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Community  Detection  in  Sparse  Random  Networks 


Nicolas  Verzelen1  and  Ery  Arias-Castro2 


We  consider  the  problem  of  detecting  a  tight  community  in  a  sparse  random  network.  This  is 
formalized  as  testing  for  the  existence  of  a  dense  random  subgraph  in  a  random  graph.  Under 
the  null  hypothesis,  the  graph  is  a  realization  of  an  Erdos- Renyi  graph  on  N  vertices  and  with 
connection  probability  poi  under  the  alternative,  there  is  an  unknown  subgraph  on  n  vertices 
where  the  connection  probability  is  p±  >  pq.  In  (Arias-Castro  and  Verzelen,  2012),  we  focused  on 
the  asymptotically  dense  regime  where  po  is  large  enough  that  log(l  V  (rapo)-1)  =  o(log(Ar/n)). 
We  consider  here  the  asymptotically  sparse  regime  where  po  is  small  enough  that  log(Ar/n)  = 
0(log(l  V  (npo)”1)).  As  before,  we  derive  information  theoretic  lower  bounds,  and  also  establish 
the  performance  of  various  tests.  Compared  to  our  previous  work  (Arias-Castro  and  Verzelen, 
2012),  the  arguments  for  the  lower  bounds  are  based  on  the  same  technology,  but  are  substantially 
more  technical  in  the  details;  also,  the  methods  we  study  are  different:  besides  a  variant  of  the  scan 
statistic,  we  study  other  statistics  such  as  the  size  of  the  largest  connected  component,  the  number 
of  triangles,  the  eigengap  of  the  adjacency  matrix,  etc.  Our  detection  bounds  are  sharp,  except  in 
the  Poisson  regime  where  we  were  not  able  to  fully  characterize  the  constant  arising  in  the  bound. 

Keywords:  community  detection,  detecting  a  dense  subgraph,  minimax  hypothesis  testing,  Erdos- 
Renyi  random  graph,  scan  statistic,  planted  clique  problem,  largest  connected  component. 

1  Introduction 

Community  detection  refers  to  the  problem  identifying  communities  in  networks,  e.g.,  circles  of 
friends  in  social  networks,  or  groups  of  genes  in  graphs  of  gene  co-occurrences  (Bickel  and  Chen, 
2009;  Girvan  and  Newman,  2002;  Lancichinetti  and  Fortunato,  2009;  Newman,  2006;  Newman 
and  Girvan,  2004;  Reichardt  and  Bornholdt,  2006).  Although  fueled  by  the  increasing  importance 
of  graph  models  and  network  structures  in  applications,  and  the  emergence  of  large-scale  social 
networks  on  the  Internet,  the  topic  is  much  older  in  the  social  sciences,  and  the  algorithmic  aspect 
is  very  closely  related  to  graph  partitioning,  a  longstanding  area  in  computer  science.  We  refer  the 
reader  to  the  comprehensive  survey  paper  of  Fortunato  (2010)  for  more  examples  and  references. 

By  community  detection  we  mean,  here,  something  slightly  different.  Indeed,  instead  of  aiming 
at  extracting  the  community  (or  communities)  from  within  the  network,  we  simply  focus  on  deciding 
whether  or  not  there  is  a  community  at  all.  Therefore,  instead  of  considering  a  problem  of  graph 
partitioning,  or  clustering,  we  consider  a  problem  of  testing  statistical  hypotheses.  We  observe 
an  undirected  graph  Q  =  (£,V)  with  N  :=  |V|  nodes.  Without  loss  of  generality,  we  take  V  = 
[N]  :=  {1,...,IV}.  The  corresponding  adjacency  matrix  is  denoted  W  =  (Wij)  E  {0, \}NxN, 
where  IV;  j  =  1  if,  and  only  if,  (i.  j)  E  £,  meaning  there  is  an  edge  between  nodes  i.j  E  V.  Note 
that  W  is  symmetric,  and  we  assume  that  Wu  =  0  for  all  i.  Under  the  null  hypothesis,  the 
graph  Q  is  a  realization  of  G(N,po),  the  Erdos- Renyi  random  graph  on  N  nodes  with  probability  of 
connection  po  £  (0, 1);  equivalently,  the  upper  diagonal  entries  of  W  are  independent  and  identically 
distributed  with  P(IVj  =  1)  =  po  for  any  i  ^  j.  Under  the  alternative,  there  is  a  subset  of  nodes 
indexed  by  S  C  V  such  that  P (Wj, j  =  1)  =  pi  for  any  i,j  E  S  with  i  ^  j,  while  P (IV, j  =  1)  =  Po  for 
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any  other  pair  i  7 -  j.  We  assume  that  p\  >  po,  implying  that  the  connectivity  is  stronger  between 
nodes  in  S,  so  that  S  is  an  assortative  community.  The  subset  S  is  not  known,  although  in  most 
of  the  paper  we  assume  that  its  size  n  :=  |5|  is  known.  Let  Ho  denote  the  null  hypothesis,  which 
consists  of  G(N,po)  and  is  therefore  simple.  And  let  Hg  denote  the  alternative  where  S  is  the 
anomalous  subset  of  nodes.  We  are  testing  Ho  versus  H\  :=  U|s|=rt  Hs-  We  consider  an  asymptotic 
setting  where 

N  — >  00,  n  =  n(N)  — >  00,  n/N  ^  0,  n/  log  N  — >  00,  (1) 

meaning  the  subgraph  is  small,  but  not  too  small.  Also,  the  probabilities  of  connection,  po  =  Po(N) 
and  pi  =  p\(N),  may  change  with  N  —  in  fact,  they  will  tend  to  zero  in  most  of  the  paper. 

Despite  its  potential  relevance  to  applications,  this  problem  has  received  considerably  less  at¬ 
tention.  We  mention  the  work  of  Wang  et  al.  (2008)  who,  in  a  somewhat  different  model,  propose  a 
test  based  on  a  statistic  similar  to  the  modularity  of  Newman  and  Girvan  (2004);  the  test  is  evalu¬ 
ated  via  simulations.  Sun  and  Nobel  (2008)  Rukhin  and  Priebe  (2012)  consider  a  test  based  on  the 
maximum  number  of  edges  among  the  subgraphs  induced  by  the  neighborhoods  of  the  vertices  in 
the  graph;  they  obtain  the  limiting  distribution  of  this  statistic  in  the  same  model  we  consider  here, 
with  po  and  p\  fixed,  and  n  is  a  power  of  N,  and  in  the  process  show  that  their  test  reduces  to  the 
test  based  on  the  maximum  degree.  Closer  in  spirit  to  our  own  work,  Butucea  and  Ingster  (2011) 
study  this  testing  problem  in  the  case  where  po  and  p\  are  fixed.  A  dynamic  setting  is  considered 
in  (Heard  et  ah,  2010;  Mongiovi  et  ah,  2013;  Park  et  ah,  2013)  where  the  goal  is  to  detect  changes 
in  the  graph  structure  over  time. 

Our  previous  work.  We  recently  considered  this  testing  problem  in  (Arias-Castro  and  Verze- 
len,  2012),  focusing  on  the  dense  regime  where  log(l  V  (npo)-1)  =  o(log(fV/n))  or  equivalently 
Po  >  n~l {n / N)°^ .  (For  a,  b  6  M,  a  Ab  denotes  the  minimum  of  a  and  b.)  We  obtained  information 
theoretic  lower  bounds,  and  we  proposed  and  analyzed  a  number  of  methods,  both  when  po  is 
known  and  when  it  is  unknown.  (None  of  the  methods  we  considered  require  knowledge  of  p\.)  In 
particular,  a  combination  of  the  total  degree  test  based  on 

W:=  W*J>  (2) 

1  <i<j<N 


and  the  scan  test  based  on 


W*:=  max  WS,  Ws  :=  £  Whj  ,  (3) 

|s|=n 

was  found  to  be  asymptotically  minimax  optimal  when  po  is  known  and  when  n  is  not  too  small, 
specifically  n/logN  — >  00.  This  extends  the  results  that  Butucea  and  Ingster  (2011)  obtained 
for  po  and  p\  fixed  (and  po  known).  In  that  same  paper,  we  also  proposed  and  studied  a  convex 
relaxation  of  the  scan  test,  based  on  the  largest  n-sparse  eigenvalue  of  W2,  inspired  by  related  work 
of  Berthet  and  Rigollet  (2012). 

Also  in  (Arias-Castro  and  Verzelen,  2012),  we  separately  considered  the  very  special  case  where 
pi  =  1,  meaning  that  the  anomalous  subgraph  S'  is  a  clique.  This  is  the  case  that  Sun  and  Nobel 
(2008)  consider  motivated  by  a  data  mining  application.  We  confirmed  the  intuition  that  the  clique 
test,  which  is  based  on  the  size  of  the  largest  clique,  is  asymptotically  minimax  optimal.  We  note 
that  the  special  case  where  po  =  1/2  and  p\  =  1  is  often  called  the  ‘planted  clique  problem’  and  has 
received  substantial  attention  in  recent  times  (Alon  et  al.,  1998;  Dekel  et  al.,  2011;  Feige  and  Ron, 
2010),  where  the  main  goal  has  been  on  designing  polynomial-time  algorithms  that  can  detect  (or 
even  extract)  the  clique  with  high  confidence.  No  such  algorithm  is  known  when  the  clique  is  of 
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size  n  =  o(VN),  even  though  in  this  setting  the  clique  test  can  detect  cliques  of  size  n  >  c  log  N  if 
c  >  2/ log  2  is  fixed. 

Continuing  our  work,  in  the  present  paper  we  focus  on  the  sparse  regime  where 


—  log(npo)  >  colog(iV/n)  for  some  constant  co  >  0. 
Obviously,  (4)  implies  that  npo  <  1.  We  define 

X0  =  Np0,  Ai  =  npi, 


(4) 

(5) 


and  note  that  Ao  and  Ai  may  vary  with  N . 

This  paper.  Compared  to  our  previous  work  (Arias-Castro  and  Verzelen,  2012),  the  derivation 
of  the  various  lower  bounds  here  rely  on  the  same  general  approach.  Let  G(N,pq-,  niPi )  denote  the 
random  graph  obtained  by  choosing  S  uniformly  at  random  among  subsets  of  nodes  of  size  n, 
and  then  generating  the  graph  under  the  alternative  with  S  being  the  anomalous  subset.  When 
deriving  a  lower  bound,  we  first  reduce  the  composite  alternative  to  a  simple  alternative,  by  testing 
Hq  :  G(N,po)  versus  Hi  :=  G(N,pQ-,n,pi).  Let  L  denote  the  corresponding  likelihood  ratio, 

i-e-,  L  =  ffTEisi =n  Lg ,  where  Lg  is  the  likelihood  ratio  for  testing  Hq  versus  Hg.  Then 
these  hypotheses  merge  in  the  asymptote  if,  and  only  if,  L  — >  1  in  probability  under  Hq.  A 
variant  of  the  so-called  ‘truncated  likelihood’  method,  introduced  by  Butucea  and  Ingster  (2011), 
consists  in  proving  that  Eo(L)  — >  1  and  Eo(L2)  — >  1,  where  L  is  a  truncated  likelihood  of  the  form 
i=C)  E,sn  nLgtrs,  where  Ts  is  a  carefully  chosen  event.  An  important  difference  with  our 
previous  work  is  the  more  delicate  choice  of  Ts,  which  here  relies  more  directly  on  properties  of 
the  graph  under  consideration.  We  mention  that  we  use  a  variant  to  show  that  Hq  and  H\  do 
not  separate  in  the  limit.  This  could  be  shown  by  proving  that  the  two  graph  models  G(N,pq) 
and  G(N,pQ-,n,pi)  are  contiguous.  The  ‘small  subgraph  conditioning’  method  of  Robinson  and 
Wormald  (1992,  1994)  —  see  the  more  recent  exposition  in  (Wormald,  1999)  —  was  designed  for 
that  purpose.  For  example,  this  is  the  method  that  Mossel  et  al.  (2012)  use  to  compare  a  Erdos- 
Renyi  graph  with  a  stochastic  block  model3  with  two  blocks  of  equal  size.  This  method  does  not 
seems  directly  applicable  in  the  situations  that  we  consider  here,  in  part  because  the  second  moment 
of  the  likelihood  ratio  L  tends  to  infinity  at  the  limit  of  detection. 

Again  compared  to  our  previous  work  (Arias-Castro  and  Verzelen,  2012),  the  methods  we  pro¬ 
pose  and  study  here  are  different.  Although  the  total  degree  test  (2)  remains  a  contender,  scanning 
over  subsets  of  size  exactly  n  as  in  (3)  does  not  seem  to  be  optimal  anymore,  all  the  more  so  when 
Pq  is  small.  Instead,  we  scan  over  subsets  of  a  wider  range  of  sizes,  using 


Wx 

vr  n 


n 

sup 

k=n/uN 


k 


(6) 


where  un  =  loglog(V/n).  We  call  this  the  broad  scan  test. 

Our  results.  In  analogy  with  our  previous  results  in  (Arias-Castro  and  Verzelen,  2012),  we  find 
that  a  combination  of  the  total  degree  test  (2)  and  the  broad  scan  test  based  on  (6)  is  asymptotically 
optimal  when  Ao  — >  oo,  in  the  following  sense.  Reparameterize  Ao  =  ( N/n)a  with  0  <  a  <  1  —  the 
case  Ao  >  N/n  being  settled  in  (Arias-Castro  and  Verzelen,  2012)  —  and  consider  n  =  NK  with 
0  <  k  <  1.  Then,  if  k  >  the  total  degree  test  is  asymptotically  powerful  when  Ai  3> 
and  the  two  hypotheses  merge  asymptotically  when  Ai  <C  When  k  <  4±g5  that  is  for 

3  This  is  a  popular  model  of  a  network  with  communities,  also  known  as  the  planted  partition  model.  In  this  model, 
the  nodes  belong  to  blocks:  nodes  in  the  same  block  connect  with  some  probability  pin,  while  nodes  in  different  blocks 
connect  with  probability  pa ut  ■ 
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Table  1:  Detection  boundary  and  near-optimal  algorithms  in  the  regime  Ao  =  ( N/n)a  with  0  <  a  < 
1  and  n  =  NK  with  0  <  k  <  1.  Undetectable  means  that  that  the  hypotheses  merge  asymptotically, 
while  detectable  means  that  there  exists  an  asymptotically  powerful  test.  Here,  a  -<  b  (resp.  a  >~  b), 
means  that  there  exists  a  positive  constant  C  such  that  a  <  Cb  (resp.  a  >  Cb). 


K 

^  l+q 

K  ^  2 +a 

K  \  l+a 

K  ^  2+o 

Undetectable 

Detectable 

Ai  -<  (1  —  a)^1;  Exact  Eq.  in  (54) 
Ai  >-  (1  —  a)^1;  Exact  Eq.  in  (13) 

\  //  A(1+“)/2 

^1  ^  77I+C* 

\  ^  N(1+a'l/2 

M  ni+a 

Optimal  test 

Broad  Scan  test 

Total  Degree  test 

Table  2:  Detection  boundary  and  near-optimal  algorithms  in  the  regimes  Ao  — >  oo  and  Ao  — >  0  and 
n  =  NK  with  0  <  k,  <  1/2.  For  1/2  <  k  <  1,  the  detection  boundary  accurs  at  Ai  x  lV1//2/n2  and 
is  achieved  by  the  total  degree  test. 


Undetectable 

Detectable 

Optimal  test 

i « Ao « (f y(1) 

lim  sup  Ai  <  1 

lim  inf  Ai  >  1 

Broad  Scan  test 

No(i)  <  A0  -  o(l) 

lim  sup  J  >  k 

i°gOo  ) 

lim  inf  log^Al  J  <  k 
i°sOo  ) 

Largest  CC  test 

smaller  n,  there  exists  a  sequence  of  increasing  functions  x/j n  (defined  in  Theorem  1)  such  that  the 
broad  scan  test  is  asymptotically  powerful  when  liminf(l  —  a)ipn(Xi)  >  1  and  the  hypotheses  merge 
asymptotically  when  limsup(l  —  a)ipn(\i)  <  1.  See  Table  1  and  the  first  line  of  Table  2  for  a  visual 
summary. 

When  1V_0+  <  Ao  <  (lV/n)0+  and  n  =  NK  with  1/2  <  n  <  1,  the  total  degree  test  is  optimal, 
in  the  sense  that  it  is  asymptotically  powerful  for  A2/Ao  n2/N ,  while  the  hypotheses  merge 
asymptotically  —  meaning  all  tests  are  asymptotically  powerless  —  for  A2/Ao  <C  n2/N.  This  is 
why  we  assume  in  the  remainder  of  this  discussion  that  n  =  NK  with  0  <  k  <  1/2. 

The  Poissonian  regime  where  Ao  and  Ai  are  considered  as  fixed  is  depicted  on  Figure  1.  When 
Ai  >  1,  the  broad  scan  test  is  asymptotically  powerful.  When  Ao  >  e  and  Ai  <  1,  no  test  is  able 
to  fully  separate  the  hypotheses.  In  fact,  when  Ao  is  bounded  from  above,  and  Ai  is  bounded  from 
below  away  from  0,  then  the  test  based  on  the  number  of  triangles  has  some  nontrivial  power, 
implying  that  the  two  hypotheses  do  not  completely  merge  in  this  case.  The  case  where  Ao  <  e  is 
not  completely  settled.  No  test  is  able  to  fully  separate  the  hypotheses  if  Ai  <  y/X^/e.  The  largest 
connected  component  test  is  optimal  up  to  a  constant  when  Ao  <  1  and  a  test  based  on  counting 
subtrees  of  a  certain  size  bridges  the  gap  in  constants  for  1  >  Ao  <  e,  but  not  completely.  When 
Ao  is  bounded  from  above  and  Ai  =  o(l),  the  two  hypotheses  merge  asymptotically.  Finally,  when 
Ao  — >  0,  the  largest  connected  component  test  is  asymptotically  optimal  (See  Table  2). 

We  also  discuss  tests  that  can  be  computed  in  polynomial  time.  Besides  the  total  degree  test,  the 
test  based  on  the  largest  connected  component  and  the  number  of  triangle  test,  already  mentioned, 
we  discuss  the  maximum  degree  test,  a  test  based  on  counting  simple  cycles  of  given  (small)  length, 
and  also  some  spectral  methods.  We  find  that,  in  the  regime  where  Ao  — >  oo,  no  test  seems  to  come 
close  to  the  broad  scan  test. 

Content.  The  remaining  of  the  paper  is  organized  as  follow.  In  Section  2  we  introduce  some 
notation  and  some  concepts  in  probability  and  statistics,  including  concepts  related  to  hypothesis 
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Figure  1:  Detection  diagram  in  the  poissonian  asymptotic  where  Ao  and  Ai  are  fixed  and  n  =  NK 
with  0  <  k  <  1/2.  “No  powerful  test”  means  that  no  test  is  able  to  fully  separate  the  hypotheses. 


testing  and  some  basic  results  on  the  binomial  distribution,  in  Section  3  we  study  some  tests  that 
are  near-optimal  in  different  regimes.  In  Section  4  we  state  and  prove  information  theoretic  lower 
bounds  on  the  difficulty  of  the  detection  problem.  In  Section  5  we  study  various  tests  that  run  in 
polynomial-time.  In  Section  6  we  discuss  the  situations  where  po  and/or  n  are  unknown,  as  well  as 
open  problems.  Section  7  contains  some  proofs  and  technical  derivations. 

2  Preliminaries 

In  this  section,  we  first  define  some  general  assumptions  and  some  notation,  although  more  notation 
will  be  introduced  as  needed.  We  then  define  and  discuss  some  basic  concepts  on  hypothesis  testing 
and  decision  theory,  and  in  particular  describe  our  strategy  for  obtaining  information  theoretic  lower 
bounds.  Finally,  we  list  some  general  results  that  will  be  used  multiple  times  throughout  the  paper. 

2.1  Assumptions  and  notation 

We  recall  that  N  — >  oo  and  the  other  parameters  such  as  n,po,p\  may  change  with  N,  and 
this  dependency  is  left  implicit.  We  assume  that  po  is  bounded  away  from  1,  which  is  the  most 
interesting  case  by  far,  and  that  N2po  — >  oo,  the  latter  implying  that  the  number  of  edges  in  the 
network  (under  the  null)  is  not  bounded.  Similarly,  we  assume  that  n2pi  — >  oo,  for  otherwise  there 
is  a  non-vanishing  chance  that  the  community  does  not  contain  any  edges.  Unless  stated  otherwise, 
we  assume  that  n  and  po  are  both  known. 

Define 

log  Ap  m 

a  log (N/n)  ’ 

in  such  a  way  that  Po  =  jf  with  Ao  =  (y~)a-  The  dense  regime  considered  in  (Arias-Castro 
and  Verzelen,  2012)  corresponds  to  liminfa  >  1.  Here  we  focus  on  the  sparse  regime  where 
limsupa  <  1.  The  case  where  a  — »•  0  includes  the  Poisson  regime  where  Ao  is  constant. 

Recall  that  Q  =  (V,£)  is  the  (undirected,  unweighted)  graph  that  we  observe,  and  for  S  C  V, 
let  Qs  denote  the  subgraph  induced  by  S  in  Q. 
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We  use  standard  notation  such  as  a n  ~  b n  when  ajv/&Ar  — >  1;  ajsr  =  o(6jv)  when  am/bjy  — >  0; 
ajy  =  0(b]y )  when  aj\r/b]\r  is  bounded;  a n  x  6/v  when  on  =  0(bN)  and  6/v  =  O(ajv);  a]y  -<  b n  when 
there  exists  a  positive  constant  C  such  that  aw  <  Cb^  and  a at  >-  b^  when  there  exists  a  positive 
constant  C  such  that  ajv  >  Cb n-  We  extend  this  notation  to  random  variables.  For  example,  if 
An  and  Bn  are  random  variables,  then  An  ~  Bn  if  An /Bn  ^  1  in  probability. 

For  define  x+  =  x  V  0  and  X-  =  (— x)  V  0,  which  are  the  positive  and  negative  parts  of 

x.  For  an  integer  n,  let 

n,2)  _  Q  _  njn_l)  (8) 

Because  of  its  importance  in  describing  the  tails  of  the  binomial  distribution,  the  following 
function  —  which  is  the  relative  entropy  or  Kullback-Leibler  divergence  of  Bern(g)  to  Bern(p)  - 
will  appear  in  our  results: 

Hp(q)=qlog(^j  +{l-q)log(^—^\  ,  p,q<E( 0,1).  (9) 

We  let  H{q )  denote  Hpo(q). 

2.2  Hypothesis  testing 

We  start  with  some  concepts  related  to  hypothesis  testing.  We  refer  the  reader  to  (Lehmann  and 
Romano,  2005)  for  a  thorough  introduction  to  the  subject.  A  test  0  is  a  function  that  takes  W  as 
input  and  returns  (j)  =  1  to  claim  there  is  a  community  in  the  network,  and  (f>  =  0  otherwise.  The 
(worst-case)  risk  of  a  test  is  defined  as 

7 n((/>)  =  =  !)  +  max  Ps(^>  =  0)  , 

|S|=n 

where  Po  is  the  distribution  under  the  null  Hq  and  P5  is  the  distribution  under  H$,  the  alternative 
where  S  is  anomalous.  We  say  that  a  sequence  of  tests  (</>jv)  for  a  sequence  of  problems  (W^r)  is 
asymptotically  powerful  (resp.  powerless)  if  7at(</>jv)  — >  0  (resp.  — >  1).  We  will  often  speak  of  a  test 
being  powerful  or  powerless  when  in  fact  referring  to  a  sequence  of  tests  and  its  asymptotic  power 
properties.  Then,  practically  speaking,  a  test  is  asymptotically  powerless  if  it  does  not  perform 
substantially  better  than  any  method  that  ignores  the  adjacency  matrix  W,  i.e. ,  guessing.  We  say 
that  the  hypotheses  merge  asymptotically  if 

7 n  '■=  mf  7-/v(0)  ->  1  , 

<p 

and  that  the  hypotheses  separate  completely  asymptotically  if  ^*N  — >  0,  which  is  equivalent  to 
saying  that  there  exists  a  sequence  of  asymptotically  powerful  tests.  Note  that  if  liminfy^  >  0, 
no  sequence  of  tests  is  asymptotically  powerful,  which  includes  the  special  case  where  the  two 
hypotheses  are  contiguous. 

2.3  Some  general  results 

Remember  the  definition  of  the  entropy  function  in  (9).  The  following  is  a  simple  concentration 
inequality  for  the  binomial  distribution. 

Lemma  1  (Chernoff’s  bound).  For  any  positive  integer  n,  any  q,p  £  (0, 1),  we  have 


P  ( Bin{n,p )  >  qn )  <  exp  (— nHp(q )) . 


(10) 
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Here  are  some  asymptotics  for  the  entropy  function. 


Lemma  2.  Define  h{x)  =  xlogx  —  x  +  1.  For  0  <  p  <  q  <  1,  we  have 


The  following  are  standard  bounds  on  the  binomial  coefficients.  Recall  that  e  =  exp(l). 
Lemma  3.  For  any  integers  1  <  k  <  n, 


(11) 


Let  Hyp  (IV,  m,  n )  denotes  the  hypergeometric  distribution  counting  the  number  of  red  balls  in 


n  draws  from  an  urn  containing  m  red  balls  out  of  N. 

Lemma  4.  Hyp(IV,  m,n)  is  stochastically  smaller  than  Bin(n,  p),  where  p  :=  N™m  ■ 

3  Some  near-optimal  tests 

In  this  section  we  consider  several  tests  and  establish  their  performances.  We  start  by  recalling  the 
result  we  obtained  for  the  total  degree  test,  based  on  (2),  in  our  previous  work  (Arias-Castro  and 
Verzelen,  2012).  Recalling  the  definition  of  Aq  and  Ai  in  (5),  define 


(pi  ~Po)2  n4  _  (Ai  -  A 0n/N)2  n2 
Po  N2~  Ac  N 


(12) 


Proposition  1  (Total  degree  test).  The  total  degree  test  is  asymptotically  powerful  if  £  — >•  oo,  and 
asymptotically  powerless  if  f  — >  0. 

In  view  of  Proposition  1,  the  setting  becomes  truly  interesting  when  — ?-  0,  which  ensures  that 
the  naive  total  degree  test  is  indeed  powerless. 

3.1  The  broad  scan 

In  the  denser  regimes  that  we  considered  in  (Arias-Castro  and  Verzelen,  2012),  the  (standard)  scan 
test  based  on  W*  defined  in  (3)  played  a  major  role.  In  the  sparser  regimes  we  consider  here,  the 
broad  scan  test  based  on  W„,  defined  in  (6)  has  more  power.  Assume  that  liminf  Ai  >  1,  so  that 
Qs  is  supercritical  under  Hs .  Then  it  is  preferable  to  scan  over  the  largest  connected  component 
in  Qs  rather  than  scan  Qg  itself. 

Lemma  5.  For  any  A  >  1,  let  r]\  denote  the  smallest  solution  of  the  equation  r]  =  exp(A (77  —  1)). 
Let  Cm  denote  a  largest  connected  component  in  G(m,  A/m)  and  assume  that  A  >  1  is  fixed.  Then, 
in  probability,  \Cm\  ~  (1  —  rj\)m  and  Wcm  ~  f  (1  — 


Proof.  The  bounds  on  the  number  of  vertices  in  the  giant  component  is  well-known  (Van  der 
Hofstad,  2012,  Th.  4.8),  while  the  lower  bound  on  the  number  of  edges  comes  from  (Pittel  and 
Wormald,  2005,  Note  5).  □ 
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By  Lemma  5,  most  of  the  edges  of  Qg  lie  in  its  giant  component,  which  is  of  size  roughly 
(1  —  r/x^n.  This  informally  explains  why  a  test  based  on  ^  is  more  promising  that  the 

standard  scan  test  based  on  W*. 

In  the  details,  the  exact  dependency  of  the  optimal  subset  size  to  scan  over  seems  rather  intricate. 
This  is  why  in  W„.  we  scan  over  subsets  of  size  n/ugr  <  k  <  n.  (Recall  that  un  =  loglog(lV/n), 
although  the  exact  form  of  un  is  not  important.)  For  any  subset  S  C  V,  let 


WZ  s  =  max  WT  . 

TcS,\T\=k 

Note  that  W£ v  =  Wf.  defined  in  (3).  Recall  the  definition  of  the  exponent  a  in  (7). 

Theorem  1  (Broad  scan  test).  The  scan  test  based  on  Wn  is  asymptotically  powerful  if  either 


^  ,  ,  r  ■  f  (,  S  n 

Inn  sup  a  <  1  and  hmmt  (1  —  a)  sup  - - -  >  1  ; 


or 


k=n/v,N 


a  — >  0  and  lim  inf  Ai  >  1  . 


k 


(13) 

(14) 


We  shall  prove  in  the  next  section  that  the  power  of  the  broad  scan  test  is  essentially  optimal: 
if  either  lim  sup  cc  <  1  and  lim  sup  (1  —  a)  sup£=  ,  n  Kg[W£  s\/k  <  1,  or  a  — >  0  and  lim  sup  Ai  < 
1,  then  no  test  is  asymptotically  powerful  (at  least  when  n2  =  o(N),  so  that  the  total  degree 
test  is  powerless).  Regarding  (13),  we  could  not  get  a  closed- form  expression  of  this  supremum. 
Nevertheless,  we  show  in  the  proof  that 

lim  inf  sup  ^  k,S ^  >  lim  inf  ■^■'(I  +  t/Ai)  >  (15) 

k=n/uN  ™  ^ 


where  rj\  is  defined  in  Lemma  5.  Moreover,  we  show  in  Section  7  the  following  upper  bound. 


Lemma  6. 

lim  inf  sup  ^  k,S ^  <  lim  inf  —  +  1  +  x/l  +  Ai  .  (16) 

k=n/uN  ™  ^ 

Hence,  assuming  a  and  Ai  are  fixed  and  positive  ,  the  broad  scan  test  is  asymptotically  powerful 
when  (1  —  a:)4y(  1  +  r)\ , )  >  1.  In  contrast,  the  scan  test  was  proved  to  be  asymptotically  powerful 
when  (1  —  >  1  (Arias-Castro  and  Verzelen,  2012,  Prop.  3),  so  that  we  have  improved  the 

bound  by  a  factor  larger  than  1  +  rjxx  and  smaller  than  1  +  2A)(1(1  +  \J  1  +  Ai).  When  a  converges 
to  one,  it  was  proved  in  (Arias-Castro  and  Verzelen,  2012)  that  the  minimax  detection  boundary 
corresponds  to  (1  —  a)Ai/2  ~  1  (at  least  when  n2  =  o(N)).  Thus,  for  a  going  to  one,  both  the 
broad  scan  test  and  the  scan  test  have  comparable  power  and  are  essentially  optimal.  In  the  dense 
case,  the  broad  scan  test  and  the  scan  test  have  also  comparable  powers  as  shown  by  the  next  result 
which  is  the  counterpart  of  (Arias-Castro  and  Verzelen,  2012,  Prop.  3). 

Proposition  2.  Assume  that  po  is  bounded  away  from  one.  The  broad  scan  test  is  powerful  if 


lim  inf 


nH(pi) 

2  log  (N/n) 


>  1  . 


The  proof  is  essentially  the  same  as  the  corresponding  result  for  the  scan  test  itself.  See  (Arias- 
Castro  and  Verzelen,  2012). 
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Proof  of  Theorem  1.  First,  we  control  Wn  under  the  null  hypothesis.  For  any  positive  constant 
co  >  0,  we  shall  prove  that 


(1  -a)W*  >  1  +  c0]  =  o(l)  . 


(17) 


Consider  any  integer  k  6  [n/uN,n],  and  let  q k  =  2(1  +  co)/[(k  —  1)(1  —  a)].  Recall  that  k ^  = 
k(k  —  l)/2.  Applying  a  union  bound  and  Chernoff’s  bound  (Lemma  1),  we  derive  that 


W*k  >  l-^k 
1  —  a 


< 


N 


k 

<  exp 


exp 


-k^H(qk) 


k{  log(eN/k)  -  ^ H(qk )} 


We  apply  Lemma  2  knowing  that  qk/po  — >  oo,  and  use  the  definition  of  a  in  (7),  to  control  the 
entropy  as  follows 


k  —  1 


H(qk) 


k  —  1 
2 

1  +  c0 


Ok 

Po 


Qk  log 

[log(N/n)  -  log  A0  +  0(log ujv)] 


l  —  a 
(1  +  c0)\og(N/n)  , 


since  log(uTv)  =  o(log(N/n)).  Consequently, 

1  +  c0 


W*k> 


1  —  a 


-k 


<  exp[-fcc0log(!V/n)(l  +  o(l))]  , 


where  the  o(l)  is  uniform  with  respect  to  k.  Applying  a  union  bound,  we  conclude  that 
P0  (1  -  at)Wl  >  1  +  c0 


<  exp  [-fcc0log(JV/n)(l  +  o(l))]  =  o(l)  . 

k=n/uN 


We  now  lower  bound  W\  under  the  alternative  hypothesis.  First,  assume  that  (13)  holds,  so 
that  there  exists  a  positive  constant  c  and  a  sequence  of  integer  kn  >  n/u n  such  that  IE s\Wk  s]  — 
kn(  1  +  c)/(l  —  a)  eventually.  In  particular,  IE s\Wk  s\  We  then  use  (19)  in  the  following 

concentration  result  for  W£  s. 

Lemma  7.  For  any  integer  0  <  k  <  n,  we  have  the  following  deviation  inequalities 


Ps  [Wfe*S  >  Es[W*jS]  +t)<  exp 


log(2) 


t  A 


8  Es\WZ,s\ 


P5  [Wls  <  Es[Wfc*s]  ~t\<  exp 


—  l°g(2)- 


8  Es[Wls 

It  follows  for  Lemma  7  that,  with  probability  going  to  one  under  P5, 


,  Vt  >  8  (l  V  ^s[W*KS})  ;  (18) 
,  Vt  >  4yjEs[W£tS]  .  (19) 


t  Wk  Wk  s 

n  -  b  -  U  ~ 
cl'Tl  run 


l  +  c/2 
1  —  a 
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Taking  co  =  c/4  in  (17)  allows  us  to  conclude  that  the  test  based  on  w\  with  threshold  YE  is 
asymptotically  powerful. 


Now,  assume  that  (14)  holds.  Because  w\  is  stochastically  increasing  in  Ai  under  Eg,  we  may 
assume  that  Ai  >  1  is  fixed.  We  use  a  different  strategy  which  amounts  to  scanning  the  largest 
connected  component  of  Qs ■  Let  C^ax  be  a  largest  connected  component  of  Qs- 

For  a  small  c  >  0  to  be  chosen  later,  assume  that  (1  —  c)n(l  —  r/y, )  <  |C^ax|  <  (1  +  c)n(l  —  rj^) 
and  Wcs  >  (1  —  c)^1(  1  —  77^),  which  happens  with  high  probability  under  P5  by  Lemma  5.  Note 
that,  because  Ai  >  1,  we  have  rj\1  <  1,  and  therefore  |Cfiax|  x  n.  Consequently,  when  computing 
Wn  we  scan  C^ax,  implying  that 


wi  > 


Wcs 

uma 

I  CL, 


v  .  v  ,A1 ,n  1  -  c  Ai  „ 

>  (l  +  c)(l-??Al)n  -  lTc  E  +7?Al)' 


Since  c  above  may  be  taken  as  small  as  we  wish,  and  in  view  of  (17),  it  suffices  to  show  that 
Ai(l  +  77^)  >  2  .  Since  r/\  converges  to  one  when  A  goes  to  one,  we  have  limy_>i  A(1  +  r]\)  =  2. 
Consequently,  it  suffices  to  show  that  the  function  /  :  A  1— >  A(1  +  r]\)  is  increasing  on  (l,oo).  By 
definition  of  77^,  we  have  rj\  <  1/A  (since  e~A  <  1/A)  and?7'(A)  =  ijx(r]\—l) / (1— Xrj\).  Consequently, 
f'( A)  =  2  +  Hence,  f'( A)  is  positive  if  i]\  <  (2A  —  l)^1  :=  a\.  Recall  that  i]\  is  the  smallest 

solution  of  the  equation  x  =  exp[A(x  —  1)],  the  largest  solution  being  x  =  1.  Furthermore,  we  have 
x  >  exp[A(x  —  1)]  for  any  x  E  [77^,1].  To  conclude,  it  suffices  to  prove  a\  >  eATA_1i.  This  last 
bound  is  equivalent  to 


1 

2(2A  —  1) 


log(2A  -  1)  >  0  . 


_ ^2 

The  function  on  the  LHS  is  null  for  A  =  1.  Furthermore,  its  derivative  ^A-i)2  P°shive  for  A  >  1, 
which  allows  us  conclude.  □ 


Proof  of  Lemma  7.  The  proof  is  based  on  moment  bounds  for  functions  of  independent  random 
variables  due  to  Boucheron  et  al.  (2005)  that  generalize  the  Efron-Stein  inequality. 

Recall  that  Qs  =  {S,£s)  is  the  subgraph  induced  by  S.  Fix  some  integer  k  E  [0,n].  For 
any  (i.  j)  E  £s,  define  the  graph  Q^:J>  by  removing  (i.  j)  from  the  edge  set  of  Qs-  Let  be 

defined  as  Wf  but  computed  on  Qg"J\  and  then  let  IT/ =  maxTcs  m=k  ■  Observe  that 

0  <  Wf  s  —  IT/E  <  1  and  that  W/E  is  a  measurable  function  of  £g^\  the  edges  set  of  Qg’JK 
Let  T*  C  S  be  a  subset  of  size  k  such  that  Wf  s  =  Wt*  ■  Then,  we  have 

E  (Wts-Kf])<  E  (WT*  ~  WT*j))  =  WT*  =  W^s  , 


where  the  first  equality  comes  from  the  fact  that  Wt*  —  w/lY  = 

Applying  (Boucheron  et  ah,  2005,  Cor.  1),  we  derive  that,  for  any  real  q  >  2, 

iiV? 


Es{W,s-Es[^,s])  +  }]  9  <  V2 *Es[wy  +  q ; 


E5{(W/)S-Es[TT/,5])l}]1/<?  <  yj2qEs[W^s\  - 

Take  some  t  >  8(1  V  y/Eg  [B7)*  s] ) .  For  any  q  >  2,  we  have  by  Markov’s  inequality 

j2qEs[W£  s]  +  q 
ps[iT,:s>Es[w,:s]  +  t]  <  '  v 
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The  choice  q  =  |  A  32  Eg^*  ]  Is  larSer  than  2  and  leads  to  (18).  Similarly,  if  take  some  t  > 
4y/Es[W^s],  and  apply  Markov’s  inequality,  we  get 


Ps  [Wfc*s<Es[Wfc*s]-t]  < 


2gEg[W; 


fc,SJ 


The 


choice  q  =  8Es^*  ]  —  ^  leads  to  (19). 


□ 


3.2  The  largest  connected  component 

This  test  rejects  for  large  values  of  the  size  (number  of  nodes)  of  the  largest  connected  component 
in  Q.  which  we  denoted  Cmax. 


3.2.1  Subcritical  regime 

We  first  study  that  test  in  the  subcritical  regime  where  limsupAo  <  1.  Define 


h  =  A  -  1  -  log  (A)  . 


(20) 


Theorem  2  (Subcritical  largest  connected  component  test).  Assume  that  limsupAo  <  1,  loglog(fV) 
o(logn)  and  log (N)  -»  oo.  The  largest  connected  component  test  is  asymptotically  powerful 
when  either  lim  inf  Ai  >  1  or 


Ao  <  Aie1  Al  for  n  large  enough  and  lim  inf 


h0 


log(n) 


Ao  +  hi  -  A0e/Ai  log (N) 


>  1 


(21) 


If  we  further  assume  that  n2  =  o(N),  then  the  largest  connected  component  test  is  asymptotically 
powerless  when  Ai  <  1  for  all  n  and 


Ao  >  Aie1  Al  for  n  large  enough  or  lim  sup 


ho 


log(n) 


A0  +  I\i  -  A0e/Ai  log(AT) 

If  we  assume  that  both  Aq  and  Ai  go  to  zero,  then  Condition  (21)  is  equivalent  to 


<  1  . 


lim  inf 


ho  log(rc) 

hi  log(A0 


>  1  , 


(22) 


(23) 


which  corresponds  to  the  optimal  detection  boundary  in  this  setting,  as  shown  in  Theorem  4. 

The  technical  hypothesis  log  log(fV)  =  o(log  n)  is  only  used  for  convenience  when  analyzing  the 
critical  behavior  Ai  — >  1.  The  condition  T^1  log(fV)  — >  oo  implies  that  Ao  can  only  converge  to  zero 
slower  than  any  power  of  N.  Although  it  is  possible  to  analyze  the  test  in  the  very  sparse  setting 
where  Ao  goes  to  zero  polynomially  fast,  we  did  not  do  so  to  keep  the  exposition  focused  on  the 
more  interesting  regimes. 


Proof  of  Theorem  2.  That  the  test  is  powerful  when  lim  inf  Ai  >  1  derives  from  the  well-known 
phase  transition  phenomenon  of  Erdos- Renyi  graphs. 


Lemma  8.  LetCm  denote  a  largest  connected  component  o/G(m,  A/m)  and  assume  that  A  E  (0,  oo) 
is  fixed.  Then,  in  probability, 


\Cm\  ~ 


h  1  l°g  m, 

(1  -  rjx)m, 


if  A  <  1  ; 
if  A  >  1  • 
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Proof.  When  A  >  1,  see  (Van  der  Hofstad,  2012,  Th.  4.8)  —  the  result  is  actually  contained  in 
Lemma  5.  When  A  <  1,  see  (Van  der  Hofstad,  2012,  Th.  4.4,  Th.  4.5).  □ 


Hence,  under  the  null  with  limsupAo  <  1,  the  largest  connected  component  of  Q  is  of  order 
log (N)  with  probability  going  to  one.  Under  the  alternative  Hg  with  liminf  Ai  >  1,  the  graph 
Qs  contains  a  giant  connected  component  whose  size  of  order  n  with  probability  going  to  one. 
Recalling  that  log(fV)  =  o(n)  allows  us  to  conclude. 

Now  suppose  that  (21)  holds.  We  assume  that  the  sequence  Ai  is  always  smaller  or  equal  to  1, 
that  =  O  (log(n)/log(lV))  and  that  log(i^  1  V  1)  =  o(logn),  meaning  that  Ai  does  not  converge 
too  fast  to  1.  We  may  do  so  while  keeping  Condition  (21)  true  because  the  distribution  of  |Cmax| 
under  P5  is  stochastically  increasing  with  Ai,  because  lim sup  Ao  <  1,  I\1  +  Ao  —  Aoe/Ai  ~/Al(l  — Ao) 
for  Ai  — >  1,  and  because  loglog(fV)  =  olog(n). 

By  hypothesis  (21),  there  exists  a  constant  c'  >  0,  such  that 


r  :=  liminf 


_ ho  log  jn) _ 

(/Al  +  A0  -  A0e/Ai )  log(JV) 


>  1  +  d  . 


To  upper-bound  the  size  of  Cmax  under  Po,  we  use  the  following. 

Lemma  9.  LetCm  denote  a  largest  connected  component  of  G(m,  X/m)  and  assume  that  A  <  1  for 
all  m  and  logf//1  V  1]  =  o(log(m)).  Then,  for  any  sequence  um  satisfying 


lim  inf  U"l  ^x  >  \  ; 
logm 


we  have 

P(|Cm|  >  Um)  =  O(l)  ■ 


Proof.  This  lemma  is  a  slightly  modified  version  of  (Van  der  Hofstad,  2012,  Th.  4.4),  the  main 
difference  difference  being  that  A  was  fixed  in  the  original  statement.  Details  are  omitted.  □ 

Define  c  =  id  A  1  )/4.  Applying  Lemma  9,  |Cmax|  <  to  :=  4'1  log(lV)(l  +  c),  with  probability 
going  to  one  under  Po- 

We  now  need  to  lower-bound  the  size  of  Cmax  under  P5.  Define 


ko  =  (1  -  c)  log(n)  [/Al  +A0-A0e/Ai]  1  ,  k  =  \k0]  , 

1  —  Aoe/Ai 


q0  =  (1  -  c)  log(n) 


Ai  +  A0  -  A0e/Ai 


Q  =  L^oJ  • 


The  denominator  of  ko  is  positive  since  Aoe^Ai  <  1  and 

h  1  +  A0  -  A0e/Ai  >  IAl  +  e_/Ai  (l  -  e A)  =  /  _,A[  >  0 


(24) 


We  note  that  k  =  O(logn),  unless  the  denominator  of  ko  goes  to  zero,  which  is  only  possible  when 
IAl  goes  to  zero  (implying  Ai  — >  1),  in  which  case 


k  ~  log(n)[/Al(l  -  A0)]  1 


O 


I71  VI 


log(n)  =  O  [log(iV)] 


(25) 


since,  in  this  case,  (21)  implies  that  4i  =  O  (log(n)/ log(V)),  and  limsupAo  <  1  by  assumption. 
So  (25)  holds  in  any  case. 
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We  shall  prove  that  among  the  connected  components  of  Qs  of  size  larger  than  q.  there  exists 
at  least  one  component  whose  size  in  Q  is  larger  than  k.  By  definition  of  c,  we  have  liminf  k/to  > 
r(l  —  c)/(l  +  c)  >  (l  +  c')(l  —  c)/(l  +  c)  >  1,  and  the  connected  component  test  is  therefore  powerful. 
The  main  arguments  rely  on  the  second  moment  method  and  on  the  comparison  between  cluster 
sizes  and  branching  processes.  Before  that,  recall  that  to  — >  oo,  so  that  log (n)i’T1/.  >~  h)  — >  oo, 
which  in  turn  implies  I\1  =  o  (log(n)). 

Lemma  10.  Fix  any  c  >  0.  Consider  the  distribution  G(rn.  A/m)  and  assume  that  A  satisfies 

lirnsup  A  <  1,  log  [T/1  V  l]  =  o(log(m))  ,  If1  logm  — >  oo  . 

For  any  sequence  q  =  alog(m)  with  a  <  /;/1(  1  —  c),  let  Z>q  denote  the  number  of  nodes  belonging 
to  a  connected  component  whose  size  is  larger  than  q.  With  probability  going  to  one,  we  have 

Z>q  >  ml-aIx~o{l)  .  (26) 

Proof.  This  lemma  is  a  simple  extension  of  the  second  method  argument  (Equations  (4.3.34)  and 
(4.3.35))  in  the  proof  of  (Van  der  Hofstad,  2012,  Th.  4.5),  where  A  is  fixed,  while  here  it  may  vary 
with  m,  and  in  particular,  may  converge  to  1.  We  leave  the  details  to  the  reader.  Q 

Observe  that 


_ q _  <  I\i  ~  A0JAlejAi  <  1  _  A  1  -  e/Ai  +  hw^1  <  1 

(1  -  c)//;1  log(n)  ~  IXl  +  A0  -  A0e/Ai  ~  °  /Al  +  A0  -  A0e/Ai 

using  the  fact  that  xex  —  ex  +  1  >  0  for  any  x  >  0.  Thus,  we  can  apply  Lemma  10  to  Qs-  And  by 
Lemma  9,  the  largest  connected  component  of  Qs  has  size  smaller  than  2///  log(n)  with  probability 
tending  to  one.  Hence,  Qs  contains  more  than 


nl+o(l)g— Ij/Aj 

2/Ai'  log  n 


-Qh  i 


-o(log(n)) 


connected  components  of  size  larger  than  q.  (We  used  the  fact  that  log  (I/)1  V  1)  =  o(log?r).)  If 
ko  —  qo  <  1,  then  applying  Lemma  10  to  q  +  2  (instead  of  q)  allows  us  to  conclude  that  there 
exists  a  connected  component  of  size  at  least  k.  This  is  why  we  assume  in  the  following  that 
liminf  ko  —  qo  >  1.  By  definition  of  ko  and  qo,  ko  —  qo  >  1,  implies  that 

log(n)A0  >  Y~e  h  1  (JV  +  Ao  -  -V7*1)  >  fiZ~fe~IxiIe-P1 


by  (24).  Thus,  liminf  ko~qo  >  1  implies  that  for  n  large  enough  log(n)Ao  >  A  1  hr  e  and  consequently 


h0  <  0(1)  -  log(Ao)  <  o  (log(n))  +  /Al  +  log 


o(log(n)) 


(27) 


since  JAl  =  o(log(n)),  -  log(IAl)  <  o(log(n))  and  Ie\  =  O  [(e  h  -  1)  2]  =  O  [lx  2] . 

Let  i  G  X}  denote  the  collection  of  connected  component  of  size  larger  than  q  in  Qs-  For 

any  such  component  Cg  ,  we  extract  any  subconnected  component  of  size  q.  Recall  that,  with 
probability  going  to  one: 


\1\  >  n1-°V>e-qI>* 


(28) 
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For  any  node  x,  let  denote  C(x)  the  connected  component  of  x  in  Q,  and  let  C-s(x)  denote  the 
connected  component  of  x  in  the  graph  Q_s  where  all  the  edges  in  Qs  have  been  removed.  Then, 
let 

Ui  '■=  IJ  C-S(x),i€l ;  V  =  Yl<m>k}  ■ 

xec^  i£l 

Since  V  >  1  implies  that  the  largest  connected  component  of  Q  is  larger  than  k,  it  suffices  to  prove 
that  V  is  larger  than  one  with  probability  going  to  one.  Observe  that  conditionally  to  |Z|,  the 
distribution  of  (\Ui\,  i  G  Z)  is  independent  of  Qs-  Again,  we  use  a  second  moment  method  based 
on  a  stochastic  comparison  between  connected  components  and  binomial  branching  processes. 

Lemma  11  (Upper  bound  on  the  cluster  sizes).  Consider  the  distribution  G(m,  A/m)  and  a  col¬ 
lection  J  of  nodes.  For  each  k  >  \J\, 

IP  [|  Cx(zj  C(x)  |  >  k]  <  Pm,A/m  [Zi  +  . . .  +  T\j\  >  k\  , 

where  Tf,  T2, . . .  denote  the  total  progenies  of  i.i.d.  binomial  branching  processes  with  parameters  m 
and  A/m.  For  each  \  J\  <  k  <  m, 

^  [|  C(x)  |  >  k ]  >  IP  m-k,X/m  [?!  +  ...+  T\j\  >  k]  , 


where  T\ ,  T2, . . .  denote  the  total  progenies  of  i.i.d.  binomial  branching  processes  with  parameters 
m  —  k  and  A/m. 

Lemma  11  is  a  slightly  modified  version  of  (Van  der  Hofstad,  2012,  Th.  4.2  and  4.3),  the  only 
difference  difference  being  that  \  J\  =  1  in  the  original  statement.  The  proof  is  left  to  the  reader. 
The  following  result  is  proved  in  (Van  der  Hofstad,  2012,  Sec.  3.5). 

Lemma  12  (Law  of  the  total  progeny).  Let  T\, ...  ,Tr  denote  the  total  progenies  ofr  i.i.d.  branching 
processes  with  offspring  distribution  X .  Then, 

P  [Ti  +  . . .  +  Tr  =  k\  =  £  P  [Xi  +  . . .  +  Xk  =  k  -  r\  , 

where  {Xf),  i  =  1, . . .  ,k  are  i.i.d.  copies  of  of  X. 

Relying  on  these  three  lemmas,  we  control  the  conditional  expectation  and  variance  of  V. 
Lemma  13.  The  following  bounds  hold 

WS[\Ui\>k\  >  (yz~q)  qe~Xoq-1^ Mn-o(!)  , 

IZIV 

Vars[V|<5s]  <  |Z|P5[|^|>A:]  +  ^-E5[|^|l{C7i>fc}]  ,  (29) 

Ps[|tfi|>fc]  <  M\Ui\t{Ui>k}}  <  (ij^)  .  (30) 

Before  proceeding  to  the  proof  of  Lemma  13,  we  finish  proving  that  V  >  1  with  probability 
going  to  one.  Let  define  :=  e-V 9— A0(fe-g)_  Applying  Chebyshev  inequality,  we  derive 

from  Lemma  13 


V  >  \I\tikn~0^ 


{\X\ Mfe)1/2no(1) 


oPs 


IXK^/JV)1/2^1) 
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In  order  to  conclude,  we  only  to  need  to  prove  that  \T\pk  >  nc  since  (|X|//fc)1//2  /\T\(p,k/N)1/2  = 
y/N/\l\  >  1.  First,  we  consider  \T\pk.  Relying  on  (28),  we  derive 


|X|  Hk 


>  ^  ^  "  g— A0q— r//A,  ~hQ(k-q) 

\k-qj 

>  n1~°^1')  (  —  ^  e-Aogo-9oi’A1-/A0(fco-go)-2i’A0 

\ko~  qoj 

>  ^l— o(l)^— (feo— 9o)g— Aog0— fcoRj^— R0(fco— 9o) 

>  o(l)e^fcoAo— fcolq  efco— <20 

>  exp  [-k0  (A0  +  /Al  -  A0e^ )]  =  nc~° «  , 


where  we  use  (27)  and  kk°qo  =  A0  xe  /ai  in  the  third  line,  the  definition  I\0  =  Ao  —  log(Ao)  —  1  in 
the  fourth  line,  and  the  definitions  of  ko  and  qo  in  the  last  line. 


Proof  of  Lemma  13.  Consider  any  subset  J  of  node  of  size  q.  The  distribution  \Ui\  =  |  (J  j(<)  C_s(x)| 

is  stochastically  dominated  by  the  distribution  of  Z  :=  \JxejC{x)\  under  the  null  hypothesis.  Let 
Tq  be  sum  of  the  total  progenies  of  q  independent  binomial  branching  processes  with  parameters 
N  —  n  +  q  —  k  and  po-  By  Lemma  11,  we  derive 


Ps[N  >  k]  >  Po[Z  >  k]  >  Pat  -n+q-k,p0  [Tq  >  k]  >  Pat  -n+q-k,p  o  [Tq  =  k] 


Let  X\ ,  X'2-, . . .  denote  independent  binomial  random  variables  with  parameters  N  —  n  +  q  —  k  and 


Po-  Relying  on  Lemma  12  and  the  lower  bound  (*)  >  ^  >  (re)' 


-1  (  (s-r)e 


we  derive 


P./V— n+q—  k,po  [Tq  k[ 


^  ^>N-n+q-k,p0  [Ri  T  •  •  -  T  Xk  k  q\ 


qfk{N -n  +  q-  k)\k-q  _  k{N_n+q. 

k\  k  —  q  J  0  1 


—n+q—k)—k+q 


y  ttt 


q_ 

k 2 


ek(N  -  n  -  2(q  -  k))]k  q  (\  0\k  q  _Aofc_. 


k  —  q 


-X0k-kO(n/N) 


y 


y 


V-Ix0(,k-q)e-\0q 

k2 


k  —  q 


k—q 


N 


s—kO(n/N) 


k  —  q 


k—q 


3— A0ij— /a0  (k—q)  o(l) 


where  (25)  with  n  log  (IV) /IV  =  o(log(n))  in  the  last  line. 


Let  us  now  prove  (30).  The  first  inequality  is  Markov’s.  For  the  second,  by  Lemma  11,  Ui  is 
stochastically  dominated  by  Tq,  the  sum  of  the  total  progenies  of  q  independent  binomial  branching 
processes  with  parameters  N  and  po,  so  that 

N  oo  oo 

Es  [\Ui\t{Ui>k}]  =  ^PsI^  ^  r]  ^  J2¥N,P0[Tq  >  r]  =  ^rPjv,p0[f?  =  r]  . 

r=k  r=k  r=k 

We  use  Lemma  12  to  control  the  deviation  of  Tq.  Below  X±,X2,  ■  ■  ■  denote  independent  binomial 
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random  variables  with  parameter  N  and  pp. 

OO  OO 

y  r  Pjv,Po [Tq  =  r]  <  Pat,Po [Xi  +  . . .  +  Xr  =  r  -  q] 

'  r  —  q 


r=k 


r=k 

oo 

£  E 

r=k 


qex  p 


—  NrH, 


p  o 


Nr 


(31) 


by  Chernoff  inequality  since 


r  -  q  >  k-q  >  k0  -  q0  =  A0ejAi  Ao  = 
Nr  ~  Nk  ~  Nk0  ~  N  >  N  “  P° 

By  Lemma  2,  HPo(a)  >  alog(afpp)  —  a  +  pp.  Thus,  we  arrive  at 

'  r  —  q 


%  [\Ui\ ^{Ui>k}\  <  E  gexp 


r=k 
oo 


-(r  -  q)  log 


r  Ai 


o 


+  r  —  q  —  rAo 


<  q  y  exp[j4r]  ;  :=  -(r  -  <?)/Ao  -  <?A0  -  (r  -  q)  log  f  ^  ■  (32) 

r=k  A  ' 


Differentiating  the  function  Ar  with  respect  to  r,  we  get 


dAr 

dr 


=  ~I\ o  -  log 

<  -  h0  ~  log 


r ~ q \  1  r-q 

—  1  H - <  ~ho  ~  log 


r 

kp  -  go 
kp 


-  1  + 


r 

kp  -  qp 


k  —  q\  k  —  q 

'  -  1  +  — r- - 


h 


:o 


=  — Aq  —  /ai  +  Aoe/Ai  , 


which  is  negative  as  argued  below  the  definition  of  k.  Consequently,  Ar  is  a  decreasing  function 
of  r.  Define  n  as  the  smallest  integer  such  that  log((r  —  q)/r)  >  —I\0/2.  Since  limsupAo  <  1,  it 
follows  r\  =  0(q).  Coming  back  to  (32),  we  derive 


%  [\Ui\^{Ui>k}]  <  q(n  -  k)+  exp[Afe]  +  q  E  exp[^4r] 

r=r\ 
oo 

(n  -  k)+  +  y  e-(r-fe)[/A0-log((r-9)/r)] 


<  qe 


<  qe/ 


<  eAkO(k 2)  , 


Ak 


r=r  i 
oo 


(n  -  k)+  +  y 


-{r-k)IXo/2 


r=r  i 


(33) 


since  limsupAo  <  1-  From  (25),  we  know  that  k  =  0(log(N))  =  n0!1),  which  allows  us  to  prove 
(30). 

Turning  to  the  proof  of  (29),  we  have  the  decomposition 

Vars[F|£s]  <  \1\  P s[Ui  >  k]  +  y  {Fs  [\Ui\  >  k,  \Up\  >  k]  -  P|  [\Ui\  >  k}} 

<  \L\  P s[Ui  >k}  +  \1\2FS  [ \Ui\  >  k  ,  Ui  n  Up  +  0] 

+m2  {Fs  [| Ui\  >  k,  \Up\  >k,UiH  Up  =  0]  -  P|  [|C7i|  >  A:]}  . 


(34) 
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The  last  term  is  nonpositive.  Indeed, 

IPs  [\Ui\  >  k,  \Ui'\  >k  ,UiH  Ui ,  =  0]  -  P|  [\Ui\  >  k } 

N 

=  [|£7i|  =  r]  (Fs  [| Uv\  >  k,  UinUi'  =  Q  \  \Ui\  =  r]  -  Ps  [\Ui>\  >  k] ) 

r=k 

N 

<  ^P5  [|£7i|  =  r]  (P5  [\Ui,\  >  k  |  Ui  n  Uv  =  0  ,  \Ut\  =r]-¥s  [\UV\  >  k] )  , 

r=k 

where  the  last  difference  is  negative,  as  the  graph  is  now  smaller  once  we  condition  on  \Ui\  >  1  and 
Ui  n  Ui /  =  0.  Consider  the  second  term  in  (34): 


N 

Ps  [\Ui\  >k,  Uif)  U^  ±  0]  =  J>s[|C/i|  =  r]  P s[Ui  D  Uv  ±  0  |  \Ui\  =  r]  . 

r=k 


By  symmetry  and  a  union  bound,  we  derive 


P s[Ui  nUi,^<b  |  |C7i|  =  r]  <  q2Fs[y  E  C_s(x)  |  | U,\  =  r }  , 


~u\  ~[it) 

for  some  x  E  CKS  and  y  E  .  Since  the  graph  Q_g  is  not  symmetric,  the  probability  that  a  fixed 
node  z  belongs  to  C-s(x)  conditionally  to  |C_s(x)|  is  smaller  for  z  E  S  \  {z}  than  for  z  E  Sc.  It 
follows  that 


P s[y  e  C_s(x)  1 1 Ui 


r\  <  Eg 


|C-s(^)|  -  1 
N  —  1 


Since  |C_5(x)|  <  r,  we  conclude 


N 

Ps  [\Ui\  >k,  Ui  n  uv  +  0]  < 

r=k 


N 


E5[|^|lw>fc}]  • 


□ 


Let  us  continue  with  the  proof  of  Theorem  2,  now  assuming  that  Ai  <  1,  that  Condition  (22) 
holds,  and  that  n2  =  o(N).  We  assume  in  the  sequel  that  I\1  <  —  log(Ao),  meaning  that  Ai 
is  not  too  small.  We  may  do  so  while  keeping  Condition  (22)  true,  because  the  distribution  of 
| Cmax |  under  Ps  is  increasing  with  respect  to  Ai  and  because  for  I\1  =  —  log(Ao),  (22)  is  equivalent 
to  limsuplog(n)/ log(IV)  <  1,  which  is  always  true  since  n2  =  o(N).  Similarly,  we  assume  that 
I\1  =  o(log(n))  while  keeping  Condition  (22)  true  since  for  Ix1  going  to  infinity,  (22)  is  equivalent 

c 

c  >  0  such  that 

h0  log  (n 


<  1  and  since  hhog(N)  — >  oo.  By  Condition  (22),  there  exists  a  constant 


lim  sup 


A0  +  IXl  -  A0e/Ai  log(IV) 


<  1  —  c  . 


(35) 


We  shall  prove  that  with  probability  Ps  going  to  one,  the  largest  connected  component  of  Q  does 
not  intersect  S.  As  the  distribution  of  the  statistic  under  the  alternative  dominates  the  distribution 
under  the  null,  this  will  imply  that  the  largest  connected  component  test  is  asymptotically  powerless. 
Denote  A  the  event 


A  :=  {For  all  (x,  y)  E  S,  there  is  no  path  between  x  and  y  with  all  other  nodes  in  Sc}  . 
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For  any  subset  T,  denote  Ct(x)  the  connected  component  of  x  in  Qt ,  and  recall  that  C(x)  is  a 
shorthand  for  C\>(x).  By  symmetry,  we  have 

Ps[^c]  <  n2  p0[y  G  C-S(x)]  <  P0[y  G  C(x)]  , 


since  the  probability  of  the  edges  outside  Qs  under  Pg  is  the  same  as  under  Pq.  Again,  by  symmetry 


G  C(x)]  =  E0[P0[y  G  C(x))  |  \C(x)\]  <  E0 


|C(x)| 

N  —  1 


< 


(N  —  1)(1  —  A0)  ’ 


as  the  expected  size  of  a  cluster  is  dominated  by  the  expected  progeny  of  a  branching  process  with 
parameters  N  and  po  (Lemma  11)  and  the  expected  progeny  of  a  subcritical  branching  process 
having  mean  offspring  p  <  1  is  (1  —  p)~1  (Van  der  Hofstad,  2012,  Th.  3.5).  Thus, 


Ps[-4C]  =  Q(n2/N )  =  o(l)  . 


(36) 


Define 

A;:=(l-c)1/2log(lV)I-o1  .  (37) 

Since  limsupAo  <  1  and  since  loglog(lV)  =  o[log(n)],  it  follows  that  k  x  log (IV)  =  n°W.  By 
Lemma  10,  |Cmax|  is  larger  or  equal  to  k  with  probability  Ps  (and  Pq)  going  to  one.  Thus,  it  suffices 
to  prove  that  PsIV^gsIb^x)!  >  A:]  — >•  0.  Observe  that 

Ps[V*eS|C(s)|  >  k\  <nPs[{|C(x)|  >fc}n^]+Ps[ic]  , 

so  that,  by  (36),  we  only  need  to  prove  that  nPg  [{|C(x)|  >  A:}  n  _A]  =  o(l).  Under  the  event  A, 
C(x)  0  S  is  exactly  the  connected  component  Cs(x)  of  x  in  Qs-  Furthermore,  C(x)  is  the  union  of 
C-s(y)  over  y  G  Cs(x).  Consequently,  we  have  the  decomposition 

k- 1 

^5  [{|C(x)|  >  A:}  n^l]  <  Ps[|c5(x)|  >k}+^2  P5[|Cs(*)|  =  q]  Ps  [Bq  I  |Cs(x)|  =  q\  , 

q= 1 

where  Bq  :=  {|  U y^cs{x)  C-s(y)\  >  k}.  By  Lemma  11,  the  distribution  of  |Cs(x)|  is  stochastically 
dominated  by  the  total  progeny  distribution  of  a  binomial  branching  process  with  parameters 
(n,  Ai/n).  Denote  by  J  any  set  of  nodes  of  size  q.  Since,  conditionally  to  |Cg(x)|  =  q,  the  event  Bq 
is  increasing  and  only  depends  on  the  edges  outside  Qs,  we  have 


Es  [Bq I  |c5(x)|  =q\<  P0  [|  u y£j  C(y)\  >  k]  , 

which  is  in  turn,  by  Lemma  11,  smaller  than  the  probability  that  the  total  progeny  of  q  independent 
branching  processes  with  parameters  (N,\q/N)  is  larger  than  k.  Relying  on  the  law  of  the  total 
progeny  of  branching  processes  (Lemma  12)  and  Lemma  11,  we  get 

Es[|Cs(a;)|  =  q]  <  ^  P  [Bin(ng,  Ai/n)  =  q  -  1]  , 

OO 

P5  [Bq  I  |Cs(x)|  =  q]  <  ^^P[Bin(Vr,A0 /N)  =  r  -  q\  . 

r=k 

Working  out  the  density  of  the  binomial  random  variable,  we  derive 

(qn\ )pl\1-Pi)nq~q+1  ~<  . 


Es[|Cs(x)|  =q\  < 
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and  for  q  <  (1  —  Aq )k,  we  get 


¥s[Bq\\Cs(x)\ 


q]  < 


q 

kexp 


—NkHpa 


k  —  q 
Nk 


which  is  exacly  the  term  (31),  which  has  been  proved  in  (33)  to  be  smaller  than 

0(k2) 


ll _ e-ik-q)IXo-q\Q 

k 


Let  define 


BP  :=  e-L1^-^A0-(fc-^)/A0 


k 


k-£ 


k-l 


Gathering  all  these  bounds,  we  get 


fc-i 


Ps[{|C(i)|>fc}nAl]  -<  Y, 


Ai  '  Ai 

q=r(l-A0)fcl 


e~Ixiq  f  k2\ 

Wy  y 


q= i 


k3 


-<  —  [e“^i(1-A°)fc  +  W  Bq]  -<  n°^  sup  Bq  , 

Al  q=  1  ge[0;fc] 


where  we  observe  that  e  A(Afc  =  B(i_\0}k  and  we  use  k  =  n°^  and  I\1  =  o(log(n)).  By 

differentiating  log(H9)  as  a  function  of  q.  we  obtain  the  maximum 


sup  Bq  < 

<jG[0;fc] 


e  kIxo  , 


if  Aoc/ai  >  1  ; 


e  /Aifcexp  [Aofc(e/Ai  —  1)]  ,  else 


Recall  that  we  assume  Aoe/Ai  <  1  so  that 

Ps  [{|C(x)|  >  k}  n  A]  -<  n0^^-  exp  [-k  {A0  +  I\1  -  A0e/Ai  }]  -<  n~^~c^  1/2+°(1)  ; 

Ai 

by  definition  (37)  of  k  and  Condition  (35).  We  conclude  that  nFs  [{ | G7 (a;) |  >  k}  n  A]  =  o(l).  □ 


3.2.2  Supercritical  regime 

We  now  briefly  discuss  the  behavior  of  the  largest  connected  component  test  in  the  supercritical 
regime  where  liminf  Ao  >  1.  When  Ao  —  log  N  — >•  oo,  the  graph  Q  is  connected  with  probability 
tending  to  one  under  the  null  and  under  any  alternative  (Van  der  Hofstad,  2012,  Th.  5.5),  which 
renders  the  test  completely  useless.  We  focus  on  the  case  where  Ao  is  fixed  for  the  sake  of  simplicity. 
In  that  regime,  we  find  that,  in  that  case,  the  test  performs  roughly  as  well  as  the  total  degree  test 
—  compare  Proposition  1. 

Proposition  3  (Supercritical  largest  connected  component  test).  The  largest  connected  component 
test  is  asymptotically  powerful  when  Ai  >  Ao  >  1  are  fixed  and  n2/N  ->  oo. 

Proof.  We  keep  the  same  notation.  Under  Po,  we  have  |Cmax|  =  (1  —  tj\0)N  +  0{VN )  (Van  der 
Hofstad,  2012,  Th.  4.16).  Hereafter,  assume  that  we  are  under  P5.  Then,  by  the  same  token, 
lcmaxl  =  (1  -  V\0)(N  -  n)  +  0(y/N  -n)  and  |C£.J  =  (1  -  ijxfin  +  0{y/n).  Given  Gs  and  Qs ■<=, 
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the  probability  that  C^x  and  Caiax  are  connected  in  Q  is  equal  to  1  —  (1  —  p0)lcmaxl  ICLxl  \  in 
probability,  since  po |Ca)ax |  |C^ax|  x  n  in  probability.  Hence,  with  probability  tending  to  one, 

|Cmax|  >  l^maxl  +  |Cmax| 

=  (!  -  VXq)(n  -n)  +  OiVN)  +  (1  -  V\i)n  +  0{y/n) 

=  (1~V\0)N +  (V\o~V\i  )n  +  0(y/N), 


with  r]xQ  —  rjXi  >  0  since  Ai  >  Ao  >  1  and  rj\  is  strictly  decreasing.  Hence,  because  n  S>  x/N  by 
assumption,  the  test  that  rejects  when  |Cmax|  >  (1  —  rj\0)N  +  \{r]\0  —  rjx^n.  fij 

When  Aq  >  1  is  fixed,  the  largest  connected  component  is  of  size  |Cmax|  satisfying 


|Cmax|  JL  VXo)N  Af(0, 1),  under  P0  , 

by  (Van  der  Hofstad,  2012,  Th.  4.16),  while  |Cmax|  increases  by  at  most  n  under  the  alternative,  so 
the  test  is  powerless  when  n  =  o(x/N). 


3.3  The  number  of  fc-trees 

We  consider  the  test  that  rejects  for  large  values  of  Nj?ee,  the  number  of  subtrees  of  size  k.  This  test 
will  partially  bridge  the  gap  in  constants  between  what  the  broad  scan  test  and  largest  connected 
component  test  can  achieve  in  the  regime  where  Ao  is  constant.  Recall  the  definition  of  lx  in  (20). 

Theorem  3.  Assume  that  X\  and  Aq  are  both  fixed,  with  0  <  i/Ao/e  <  Ai  <  1,  and  that 


lim  sup 


log  (N/n2) 
log  n 


< 


e 


(38) 


Then  there  is  a  constant  c  >  0  such  that  the  test  based  on  Nj?ee  with  k  =  clogn  is  asymptotically 
powerful. 


Thus,  even  in  the  supercritical  Poissonian  regime  with  1  <  Ao  <  e,  there  exist  subcritical 
communities  Ai  <  1  that  are  asymptotically  detectable  with  probability  going  to  one.  The  condition 
Ai  >  y/Xo/e  will  be  shown  to  be  minimal  in  Theorem  5.  Condition  (38)  essentially  requires  that 
n2/N  does  not  converge  too  fast  to  zero.  In  particular,  when  n  =  NK,  (38)  translates  into  an  upper 
bound  on  k.  We  show  later  in  Theorem  5  that  such  an  upper  bound  is  unavoidable,  for  when  k  is 
too  small,  no  test  is  asymptotically  powerful.  Nevertheless,  Condition  (38)  is  in  all  likelihood  not 
optimal. 


Proof  of  Theorem  3.  We  first  compute  the  expectation  of  N^ree  under  Po  using  Cayley’s  formula. 
Since  k2  =  o{n )  =  o(N)  and  k  — >  oo,  we  derive 


y,  Po  [Gc  is  a  tree] 

\C\=k 


(Nky-2Pt\i-P0r^ 


N(  X0e)k 


1 

x/2irk5/2Xo 
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where  we  used  the  fact  any  fc-tree  has  exactly  k—  1  edges.  The  last  line  comes  from  an  application 
of  Stirling’s  formula.  We  then  bound  the  variance  of  NjXee  under  Po  in  the  following  lemma,  whose 
lengthy  proof  is  postponed  to  Section  7.3. 

Lemma  14.  When  Xq  <  e,  we  have 

Var0[iV£ree]  -<  -^(eA0 )ke2kV^  . 

K  A0 

By  Chebyshev’s  inequality,  under  Pq, 

Njfee  =  E0  JVfee  +  Q(  Var0(iV*ree))1/2  . 


Fix  S  C  V  of  size  |S|  =  n,  and  let  q  be  an  integer  between  1  and  k  chosen  later.  We  let  ec 
denote  the  number  of  k- trees  in  C/gc,  and  let  Nj?geq  as  the  number  of  subsets  C  of  size  k  such  that 
\C  n  S\  =  q  and  both  Qcrs  and  Qc  are  trees.  We  have  Njfee  >  NjXg ®  Therefore,  by 

Chebyshev’s  inequality,  under  Pg 

JVf“  >  ES  +  ES  (NlJ,)  +  0(  Vars(jv;/|t))1/2  +  0(  Vars(JV,“jy)1/2  . 

Noting  that  Qsc  ~  G(N  —  n,po),  and  letting  A'0  =  (N  —  n)po,  Lemma  14  implies  that 

Var sWtjc]  ~<  ^^(eX'0)ke2kVWe  „  ^-(eX0)ke2 k^e  , 
because  nk  =  o(N).  Thus,  we  only  need  to  show  that,  for  a  careful  choice  of  q, 


Esrwsy 

» 

Eo[iVj]-Eg[lV01]  , 

(39) 

> 

Varg[AT^]  , 

(40) 

E|[JV^] 

> 

^-{eX0)ke2k^~e  >-  Var0[lV*ree]  . 
kXo 

(41) 

From  now  on,  let  q  =  k  —  • 

We  use  the  following  lemma,  whose  lengthy  proof  is  postponed  to  Section  7.4. 
Lemma  15.  When  q  =  k  —  Ly^’J  >  we  have 


Eg  [IV, 


\fe-l  2k-q 

W  y  n~  p -  ^  n(eAi) 


Ta -k 

o  X\e 


Ai  k3 


(42) 


and 


Var s[Nk™q]  -<  nk  X1 


2k-q-leAk-2qe2^q 


k^n2  2k— 2  \  „4k-2q 

~^TXi  A0e 


We  first  prove  (39),  bounding 


E0[N^\-Es[Nlsc] 


{N k  n))kk~2pk0-Hi-Pof2)-k+1 


-<  n(Xoe)kk  V2  , 


(43) 
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since  [1  —  (n  +  k)/N]k  =  1  +  kn/N  +  o(kn/N )  by  the  fact  that  k  =  o{n)  and  kn  =  o(N).  We  also 
used  Stirling’s  formula  again.  Using  this  bound  together  with  (42),  we  derive 


y 


Ao  ( Ai 


E0[Njfee]  -Es[Nlsc]  k1/2X1  \X0 


'  k4°- 
e  xie  = 


Ao 


k1/2Xi 


exp 


kl. 


oo 


since  Ao  and  Ai  are  fixed  such  that  Ao/Aie  <  1,  implying  that  I  a0  >  0  is  fixed. 


Second,  we  prove  (40).  Using  (42)  and  (43),  we  have 


Vars[iV£sy  k\_„  ,vS.  k 13 


tree  1 


n 


k8 

-<  —  exp 


n 


+ 


,13 


N  ’ 


and  the  RHS  goes  to  0  as  long  as 


lim  sup 


< 


log(n)  2(1-^)/^' 

e 

Finally,  we  prove  (41).  Using  Lemma  14  and  (42),  we  have 


Var0[lVfee]  ,  Nk5  (  X\  \k  2k J^-4k+2q 


4K 


ree  1 

k,S,qi 


-< 


-< 


n * 
Nk 5 


eAo 


n * 


exp 


2k  (I  I  \ Q  ) 


Note  that  I  /A(.  —  I  a0  <  0  is  hxed,  since  our  assumptions  imply  that  ^  <  N  <  1  and  the 


'  Aq 


function  I\  is  decreasing  on  (0, 1).  Thus,  the  RHS  above  goes  to  0  as  long  as 

k  1 


lim  inf 


> 


log (N/n2)  2  {Iko_-I  fe) 


n 


3.4  The  number  of  triangles 

We  recall  that  this  test  is  based  on  the  number  T  of  triangles  in  Q .  This  is  an  emblematic  test 
among  those  based  on  counting  patterns,  as  it  is  the  simplest  and  the  least  costly  to  compute.  As 
such,  the  number  of  triangles  in  a  graph  is  an  important  topological  characteristic,  with  applications 
in  the  study  of  real-life  networks.  For  example,  Maslov  et  al.  (2004)  use  the  number  of  triangles  to 
quantify  the  amount  of  clustering  in  the  Internet. 

Proposition  4.  The  triangle  test  is  asymptotically  powerful  if  either 


or 


lim  sup  Aq  <  oo  and  Ai  — >  oo  ; 


A2  /  a  \  2/3 

lim  inf  Ao  >  0  ,  Ao  <  N/ n  and  -A  >  1  V  =  ) 

A0  \VNj 


(44) 

(45) 
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When  Ao  and  Ai  are  fixed,  T  converges  in  distribution  towards  a  Poisson  distribution  with  parameter 
Aq/6  under  the  null  and  (Aq  +  Af)/6  under  the  alternative  hypothesis.  In  particular,  the  test  is  not 
asymptotically  powerless  if 

lim  sup  Ao  <  oo  and  lim  inf  Ai  >  0  .  (46) 

Proof  of  Proposition  f.  Let  T  be  the  number  of  triangles  in  Q.  For  S  C  V,  let  Tg  denote  the 
number  of  triangles  in  Qg.  We  have  T  >  Tg c  +  Tg. 

The  following  result  is  based  on  (Bollobas,  2001,  Th.  4.1,  4.10).  We  use  it  multiple  times  below 
without  explicitly  saying  so. 

Lemma  16.  LetTm  be  the  number  of  triangles  in  G(m,  X/m).  If  X  is  fixed,  then  Tm  Poisson(A3/6). 
If  A  — »•  oo  with  log  A  =  o(logm),  then  =>  jV(0, 1)  where  p  :=  ET  =  (™)  (^)3  ~  ^ . 

Assume  that  (44)  holds.  Applying  Lemma  16,  T  — >  0  under  Pq,  while  T  >  Tg  — >  oo  under  Pg. 
(For  the  latter,  we  use  the  fact  that  T  is  stochastically  increasing  in  Ai.) 

Assume  that  Ao  and  Ai  are  fixed.  Applying  Lemma  16,  T  =>■  Poisson(Ag/6)  under  Pq,  while 
T  =>  Poisson(Ao/6)  under  Pq,  while  under  Pg,  Tgc  +  Tg  =>■  Poisson((Ao  +  Ai )3/ 6)  since  Xgc  ~ 
G(N  —  n, po)  and  Ts  ~  G(n,p\)  are  independent,  and  n  =  o(N).  Define  Tg,gc  :=  T  —  Tg  —  Tgc  as 
the  number  of  triangles  in  Q  with  nodes  both  in  S  and  Sc.  We  have 

Es  [Ts,gc]  <  N2npl  +  n2NPlP20  <  ^A„  +  ^AXA5  =  o(l)  , 

so  that  Tg.gc  =  ops(l),  and  by  Slutsky’s  theorem,  T  Poisson((Ag  +  Ai)3/6)  under  Pg. 

Assume  that  (46)  holds.  By  considering  a  subsequence  if  needed,  we  may  assume  that  Ao  <  00 
is  fixed.  And  since  T  is  stochastically  increasing  in  Ai  under  the  alternative,  we  may  assume  that 
Ai  >  0  is  fixed.  We  have  proved  above  that  T  =>  Poisson(AQ/6)  under  Pq,  T  =>  Poisson (Aq/6+ A3 /6) 
the  alternative;  hence  the  test  {T  >  1}  has  risk 

P0(T  >  1)  +  Pg(Tg  =  0)  ->•  1  -  e"Ao/6  +  e-Ao/6-Ai/6  <  1  . 


Finally,  assume  that  (45)  holds.  Using  Chebyshev’s  inequality,  to  prove  that  the  test  based  on 
T  is  powerful  it  suffices  to  show  that 


Eg  T  -  E0  T 


00 


VVarg(T)  V  Var 0(T) 

Straightforward  calculations  show  that  EoT  =  (^)p(n  and 

Varo(T)  =  [3(N  -  3)(1  -  p0)p2  +  (1  -  p3)] 

x  N4p50  +  N3p30- 

And  carefully  counting  the  number  of  triplets  with  2  or  3  vertices  in  S  gives 


N 


Po 


EgT  = 


N  —  n 


Po  + 


n\  ( N  —  n 

1 


Po  + 


n\  ( N  —  n 
1 


P0P1  + 


71 


Pi 


(47) 


while  counting  pairs  of  triplets  with  a  certain  number  of  vertices  in  S,  shared  or  not,  we  arrive  at 
the  rough  estimate 


Varg(T)  x  N4p^  +  n2N2p^pi  +  n3Np%pl  +  nAp\  +  Eg  T  . 
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Note  that 

EST-E0T  =  (2)  (  ¥  1  ")*•  ~Po)+  Q  (p?  -  Po) 

x  n2Oi  -  p0)  [^Po  +  nPi] 

y  Nti2PqPi  +  n3p\  ,  (48) 

since  by  condition  (45),  npo  <  1  and  np\  =  Ai  1.  and 

Var0(T)  -C  Var g(T)  x  N4p^  +  n2N2plpi  +  n3Nplp\  +  n4pf  +  iV3pg  +  n3p\  . 

We  only  need  to  prove  that  the  square  root  of  this  last  expression  is  much  smaller  than  (48).  Since 
(npi)2  3>  Npo  and  np\  — >  00,  we  first  derive  that 

r?Nplp\  +  nAp\  +  N3pl  +  n3p\  =  o  [(npi)6]  . 

Similarly,  we  get  n2N2pQp\  =  o  \nAp\ N2p^\ .  Finally,  (45)  entails  that  Ay  2>  Ao(Ao /VN)2//3  which 
is  equivalent  to  IV4pg  =  o  [(npi)6] .  □ 

4  Information  theoretic  lower  bounds 

In  this  section  we  state  and  prove  lower  bounds  on  the  risk  of  any  test  whatsoever.  In  most  cases, 
we  find  sufficient  conditions  under  which  the  null  and  alternative  hypotheses  merge  asymptotically, 
meaning  that  all  tests  are  asymptotically  powerless.  In  other  cases,  we  find  sufficient  conditions 
under  which  no  test  is  asymptotically  powerful. 

To  derive  lower  bounds,  it  is  standard  to  reduce  a  composite  hypothesis  to  a  simple  hypothesis. 
This  is  done  by  putting  a  prior  on  the  set  of  distributions  that  define  the  hypothesis.  In  our  setting, 
we  assume  that  po  is  known  so  that  the  null  hypothesis  is  simple,  corresponding  to  the  Erdos- Renyi 
model  G(N,po).  The  alternative  H\  :=  (J|Si_n.Hg  is  composite  and  ‘parametrized’  by  subsets  of 
nodes  of  size  n.  We  choose  as  prior  the  uniform  distribution  over  these  subsets,  leading  to  the 
simple  hypothesis  H\  comprising  of  G(N,po',n,pi)  defined  earlier.  The  corresponding  risk  for  Hq 
versus  H 1  is 

7tf(0)  =  po0  =  1)  +  ~7m  Y  =  °)  • 

\nj  |S|=n 

Note  that  7at(</>)  >  jjv  (</>)  for  any  test  q b.  Our  choice  of  prior  was  guided  by  invariance  consid¬ 
erations:  the  problem  is  invariant  with  respect  to  a  relabeling  of  the  nodes.  In  our  setting,  this 
implies  that  ^*N  =  7^,  or  equivalently,  that  there  exists  a  test  invariant  with  respect  to  permutation 
of  the  nodes  that  minimizes  the  worst-case  risk  (Lehmann  and  Romano,  2005,  Lem.  8.4.1).  Once 
we  have  a  simple  versus  simple  hypothesis  testing  problem,  we  can  express  the  risk  in  closed  form 
using  the  corresponding  likelihood  ratio.  Let  Pi  denote  the  distribution  of  W  under  H\ .  meaning 
G(N,po-,n,pi).  The  likelihood  ratio  for  testing  Po  versus  Pi  is 

-k  =  jm  >  (49) 

|S'|=n 


where  L$  be  the  likelihood  for  testing  Po  versus  Pg.  Then  the  test  (ft* 
that  minimizes  7  at,  and 


7iv(<n=7^  =  l-2Eo|£-l| 


{L  >  1}  is  the  unique  test 
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For  each  subset  S  C  V  of  size  n,  let  Tg  be  an  event,  i.e. ,  a  subset  of  adjacency  matrices,  and  define 
the  truncated  likelihood  as 

^  =  i  Eislrs'  (5°) 

In/  |5|=n 

We  have 

IEo  |  L  —  1|  <  Eo  |  L  —  lj  -)-  Eo(T  —  A) 

<  ^Eq [Z2]  -  1  +  2(1  -  E0 [L])  +  (1  -  Eq[L\)  , 

using  the  Cauchy-Schwarz  inequality  and  the  fact  that  Eo  L  =  1  since  it  is  a  likelihood.  Hence,  for 
all  tests  to  be  asymptotically  powerless,  it  suffices  that  Eo[Z2]  <  1  +  o(l)  and  Eo[Z]  >  1  +  o(l). 
Note  that 

Eo [L\  =  E  Fs(T3)  ■ 

\n)  |5|=n 

In  all  our  examples,  Ps(rs)  is  independent  of  S,  so  that  Eq[F]  >  l+o(l)  is  equivalent  to  Ps(r,s)  — >  1. 


4.1  All  tests  are  asymptotically  powerless 

We  start  with  some  sufficient  conditions  under  which  all  tests  are  asymptotically  powerless.  Recall 
a  in  (7)  and  £  in  (12).  We  require  that  £  — >•  0  below  to  prevent  the  total  degree  test  from  having 
any  power  (see  Proposition  1). 


Theorem  4.  Assume  that  £  — >  0. 
following  situations: 

Aq  — >  0, 


Then  all  tests  are  asymptotically  powerless  in  either  of  the 

Ai  — >•  0,  lim  sup  <  1  ;  (51) 

h i  log  N 


0  <  lim  inf  Ao  <  lim  sup  Ao  <  oo,  Ai  — >  0  ; 
Aq  — >  oo  with  a  — >  0,  lim  sup  Ai  <  1  ; 


0  <  lim  inf  a  <  lim  sup  a  <  1 , 


lim  sup  (1  —  a)  sup 
k=n/uN 


Es[Wt 


k,Si 


<  1 


(52) 

(53) 

(54) 


We  recall  here  the  first  few  steps  that  we  took  in  (Arias-Castro  and  Verzelen,  2012)  to  derive 
analogous  lower  bounds  in  the  denser  regime  where  lim  inf  a  >  1.  We  start  with  some  general 
identities.  We  have 


with 


Ls  :=  exp(0H/5'  —  A (9)n^)  , 

(55) 

(56) 

and 

A(0)  :=  log(l  ~p0  +p0e9)  , 


which  is  the  moment  generating  function  of  Bern(po)- 
In  all  cases,  the  events  Ts  satisfy 


Tg  C  { Wt  <  Wk,  VT  C  S  such  that  \T\  =  k}  , 

fc>/cmin 


(57) 
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where  km ;n  and  Wk  vary  according  to  the  specific  setting. 

To  prove  that  Eq  L2  <  1  +  o(l),  we  proceed  as  follows.  We  have 


Eo  [L2]  =  — E  E  E°  (LSlLs2lrSltr 

In)  |Si|=n  |S2|=n 

1 


s2 


(NY  E  E  E° 

Vn/  |Si|=ra  |S2|=ra 


exp 


(e(WSl  +  ITs2)  -  2A (d)n^)  lrSlnrS2 


Define 


wSxT  =  \  E  > 

i€S,j£T 


and  note  that  Ws  =  Ws'xS-  We  use  the  decomposition 


WSi  +  Ws2  =  WSlX(s1\S2)  +  W/S,2x(52\Si)  +  2W/s,inS2  , 
and  the  independence  of  the  random  variables  on  the  RHS  of  (58),  to  get 

E0  (exp  (0(WSl  +  WS2 )  -  2A (0)n^)  lrSlnrS2)  <  I  ■  II  •  III  , 

where  K  =  |Si  fl  S2I, 


I:=E0 


exp 


(oWSlX(Sl\  s2) 


m 


(n  —  K)(n  +  K  —  1) 


=  1 


(58) 


(59) 


II  :=  E0 


exp  (  0Ws2x(s2\s1)  ~  -  K) (n  +  K  -  1) 


=  1  , 


III  :=  E0 


exp 


(2e\VSlns2  ~  2A (9)K^  trSlnrS2 


The  first  two  equalities  are  due  to  the  fact  that  the  likelihood  integrates  to  one. 

Assuming  that  £  ►  0,  we  prove  that  all  tests  are  asymptotically  powerless  in  the  following 

settings: 

limsupAo  <00,  A^  =  o(Aq)  ;  (60) 


Ao  — >  0,  Ai  — >  0,  limsup  <  1  n2  =  o(N ) 

I\!  log  (AT)  v  ' 

limsupAi  <1,  Ao  — >  00  ,  limsup  a  <  1  ; 


(61) 

(62) 


liminf  Ai  >  1, 


0  <  liminf  a  <  limsup  ct  <  1, 


limsup  (1  —  a)  sup 

k=n/ujsf 


Es[Wfc% 


<  1 


(63) 


This  implies  Theorem  4.  Indeed,  (60)  includes  (52).  Assume  that  (51)  holds.  Consider  any 
subsequence  n2 /N  converging  to  x  G  M+  U  {00}.  If  x  =  0,  then  (61)  holds.  If  x  /  0,  then  (  — s ►  0 
implies  that  (Ai  —  X^n/N)2 /Xq  =  o(l).  If,  in  addition,  Ai  >  2Xqu/N  ,  this  implies  that  Af/Ao  =  o(l). 
If,  otherwise,  Ai  <  2Xon/N ,  then  Af/Ao  <  4Ao (n/N)2  =  o(l)  since  Ao  =  o(l).  Thus,  in  both  cases, 
(60)  holds.  Finally,  (62)  includes  (53)  and  also  (54)  when  limsupAi  <  1,  while  (63)  includes  (54) 
when  liminf  Ai  >  1.  We  note  that  (63)  implies  that  limsupAi  <  00  because  of  (15). 
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4.1.1  Proof  of  Theorem  4  under  (60) 

The  arguments  here  are  very  similar  to  those  used  in  (Arias-Castro  and  Verzelen,  2012),  except  for 
the  choice  of  events  Tg.  Define 

Tg  :=  {Qs  is  a  forest}  . 

When  Tg  holds,  for  any  T  C  S,  Gt  is  also  a  forest,  and  since  any  forest  T  with  k  nodes  and  t 
connected  components  (therefore  all  trees)  has  exactly  k  —  t  <  k  edges,  we  have  Wt  <  \T\.  Hence, 
(57)  holds  with  wk  :=  k. 

Lemma  17.  Pg(rg)  is  independent  of  S  of  size  n,  andPg(Tg)  =  l  +  o(l). 


Proof.  The  expected  number  of  loops  of  size  k  in  Qs  under  Pg  is  equal  to 


n! 

(n  —  k)\2k 


p\< 


(64) 


Summing  (64)  over  k,  we  see  that  the  expected  number  of  loops  in  Qs  under  Pg  is  smaller  than 
Af/ (1  —  Ai)  =  o(l).  Hence,  with  probability  going  to  one  under  Pg,  Qs  has  no  loops  and  is  therefore 
a  forest.  □ 


In  order  to  conclude,  we  only  need  to  prove  that  Eo[L2]  5:  1  +  o(l).  We  start  from  (59)  and  we 
recall  that  K  =  |Si  n  52 1.  We  take  km in  as  the  largest  integer  k  satisfying. 

2  >  pj(l  -Po) 

k  -  3  “  p0(l  -  pi)2 

with  the  convention  2/0  =  oo,  so  that  km\ n  >  3.  Let  qk  =  2 /{k  —  1).  Recall  that  p  =  n/(N  —  n) 
and  define  fco  =  \bnp\,  where  b  — >  oo  satisfies  b2(  — y  0. 


•  When  K  <  A:m;n,  we  will  use  the  obvious  bound: 

III  <  E0  exp  (29WSlnS2  ~  2A (0)K^  =  exp  (a/P(2))  , 

where 

A  :=  A(20)  -  2A(0)  =  log  fl  +  • 

V  Po(l  —  Po) ) 


(65) 


•  When  K  >  kmin,  we  use  a  different  bound.  Noting  that  Tg1  nrg2  C  {H/g1nS2  —  wk},  for  any 
f,  G  (0,  29),  we  have 


III 


<  E0 

exp  (cWSlnS2  +  (2 9  -  f)wK  -  2A(9)I<^ 

<  Eq 

exp  (eWSinSa  +  (29  -  f)wK  ~  2A(9)K^ 

1 

so  that 


where 


III  <  exp  (aa^A'(2))  , 


^ k  ■= 


min 

€6  [0,20] 


A(0  +  (29  -  £)qk  -  2A (9)  . 


(66) 
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Using  the  fact  that  Eo[L2]  <  Eo[III],  we  have 

E0[L2]  <  E  exp  (aIL^2^ 

+  E  l{fc0+i<^<fcmin}  exp 


+  E 


^{fcmin+i^ftXn}  exp  (  Ak 


=  Ai  +  A2  +  H3  , 

where  the  expectation  is  with  respect  to  K  ~  Hyp (N,n,n).  By  Lemma  4,  K  is  stochastically 
bounded  by  Bin(n,p).  Hence,  using  Chernoff’s  bound  (see  Lemma  1),  we  have 

P (K  >  k )  <  P(Hyp(lV,  n,  n )  >  k )  <  P(Bin(n,  p)  >  k )  <  exp  (— nHp(k/n )) .  (67) 

•  When  K  <  ko,  we  proceed  as  follows.  If  ko  =  1,  we  simply  have 

Hi  =  F(K  <  1)  <  1  . 

If  k0  >  2,  we  use  the  expression  (65)  of  A  to  derive 

(. Pi  -  Po f  b2nA' 


Hi  <  exp  [A7'q]  <  exp 


0(1); 


=  exp  [0(b2 C)]  =  1  +  o(l) 


Po(l  ~Po)  N2 

When  ko  +  1  <  K  <  km in,  we  use  (67)  and  the  identity  (1  —  x )  log(l  —  x)  >  —x,  to  get 


k{k-  1) 

-  nHp 

2 

f  k  -  1 

to 

1 

-log  [ 

H2  <  Y2  exP  A 

k=ko~\-l 
k  ■ 

rvmin 

<  exp 

k=ko~\~l 

The  last  sum  is  equal  to  zero  if  km\n  <  ko]  therefore,  assume  that  kmin  >  ko-  For  a  >  0 
fixed,  the  function  f(x)  =  ax  —  logx  is  decreasing  on  (0, 1/a)  and  increasing  on  (1/a,  00). 
Therefore,  for  ko  +  1  <  k  <  kmin, 


A — — - log  (  —  )  <  max 

{  fc0 ,  ^-min  I 


(—) 

\npj 


i-  1 


-  log  I  10 


We  know  that  A(fco  —  1)  =  o(l),  so  that 

ko~l  ,  /  kA 


A- 


—  log  —  <  o(l)  —  log  b  — >  —00  . 


Therefore,  it  suffices  to  show  that 

i.  .  _  1 

^min  ± 


\np  ) 


A  —  log 


np 


— >  —00  . 


If  ^’min  >  3,  observe  that 


b  .  _  1 

f'min 


-A  <  1  + 


^min  3 


log  1  + 


^min  3 


(l  +  o(l))  <  -  log  3 +  o(l)  , 
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while  log(/cm;n/ (npj)  >  log(ko  /  (np))  — >  oo.  If  we  have  A:min  =  3,  then  we  have 


A  —  log 


(A) 

\npj 


<  log(pf/po)  -  log(IV/n2)  +  0(1)  <  log 


^  )  +  0(1)  — >  — oo  , 


because  of  (60). 


When  kmin  <  K  <  n,  we  need  to  bound  Ak-  Remember  the  definition  of  the  entropy  function 
Hq  in  (9),  and  that  H(q )  is  short  for  Hpo(q).  It  is  well-known  that  H  is  the  Fenchel-Legendre 
transform  of  A;  more  specifically,  for  q  E  (po,  1), 


H(q)  =  sup[<7$  -  A(0)]  =  q6q  -  A (0q) 
e>o 


(68) 


Hence,  the  minimum  of  A(£)  +  (29  —  £ )qk  —  2A (9)  over  ^  >  0  is  achieved  at  £  =  9qk  as  soon 
as  29  >  9qk.  Moreover,  by  definition  of  9  in  (56),  our  choice  of  q^,  and  the  fact  that  k  >  km in, 
we  have 

pfil-po)  2 


20  ~  °qk  =  log 


Po(l  ~Pi)2  k-  3 


>  0  . 


Hence,  we  have 

Afc  =  -H(qk) +  29qk-2A(9) 

=  2 Hp1(qk)  +  H(qk)  . 

Using  the  definition  of  the  entropy  and  the  fact  that  p$  =  o(l),  we  therefore  have 

(1  -  Pi)2 


(69) 


Afc  =  qk  log 
2 

< 


Pj 

QkPO 


+  (1  -  qk)lo g 


k-1 


log 


A lN(k  -  1) 


2Aon2 


(1  -  qk)(l-po) 
+  0(1)  , 


where  the  0(1)  is  uniform  in  k.  Hence, 


A3  <  ^2  exp 

k=kmin-\- 1 
n 

<  exp 

k=kmin-\- 1 


k  <  log 


+  log 


k ' log  ‘  a)  > + 


N(k-l) 

2  n? 

=  o( 1)  , 


-  log 


Nk 

n2 


+  0(1) 


since  Af/Ao  =  o(l). 

This  concludes  the  proof  of  Theorem  4  under  (60). 


4.1.2  Proof  of  Theorem  4  under  (61) 

Let  c  be  a  positive  constant  that  will  be  chosen  small  later  on.  Define 

fn  ■=  (1  +  c)  I 2  log (n)  . 

We  consider  the  event 

+5  =  {Gs  is  a  forest}  n  {|CmaXjS|  <  fn}  . 

When  Ts  holds,  for  any  T  C  5,  Qt  is  also  a  forest,  with  \T\  —  Wt  connected  components.  Since  the 
size  of  each  connected  component  is  at  most  fn,  there  are  at  least  |"|T|//n]  connected  components. 
Hence,  (57)  holds  with  iuk  =  k  — 
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Lemma  18.  Ps(rs)  is  independent  of  S  of  size  n,  andPs{^  s)  =  l  +  o(l). 

Proof.  This  is  a  straightforward  consequence  of  Lemmas  9  and  17.  □ 

To  conclude,  it  suffices  to  show  that  Eo[L2]  <  1  +  o(l).  For  this,  we  will  need  the  following. 

Lemma  19.  Let  Fkj  stand  for  the  number  of  forests  with  j  trees  on  k  labelled  vertices.  For  any 
k  >  2  and  any  j  <  k,  Fk  j  <  kk~2 . 

Proof.  Fix  k  >  2.  By  Cayley’s  formula,  we  have  Ff-  i  =  kk~2.  Therefore,  it  suffices  to  prove  that 
Fk,j  >  Fkj+i  for  all  j  >  1.  If  we  take  a  forest  with  j  trees  and  erase  any  of  its  k  —  j  edges,  we 
obtain  a  forest  with  j  +  1  trees.  And  there  are  exactly  kskt  such  ways  of  obtaining  a  given 
forest  with  j  +  1  trees  of  sizes  k\  <  •  •  •  <  kj+ 1.  Since 

kskt  >  k\(k  —  k\)  >  k  —  1  , 

S^t 

it  follows  that  Fk:j(k  —  j)  >  Fkj+i(k  —  1).  Thus,  Fkj  >  Ff-j+i-  Q 


Starting  from  (59),  and  using  the  fact  that,  under  r,sinrs2,  Gs1r\S2  is  a  forest  with  Ws, n.S'2  £  wr 
edges,  we  have 

E0[T2]  <  E0  (exp  (26WSlnS2  -  2A(9)K(2]^j  l{g;SinS2  is  a  forest,  wSlns2<wK})  ■ 

Note  that  the  exponential  term  is  smaller  than  1  when  |Si  n  52 1  <  1.  Recall  that  p  =  Nnfm  and 
that  A (6)  =  log  [(1  —  Po)/(l  —  pi)].  We  derive 


n  wk 


Eo[L2]-1  <  EE  P  [K  =  k,  Ws1nS2  =  i,Gs1r\S2  is  a  forest]  exp  [2 i6  —  2A (9)k^2'1] 

2(i— P2)) 


< 


A 


-< 


A 


-< 


k=2  i= 1 
n  wk 

EE 

k= 2  i=l 
n  Wk 


n\  J  JP  Pi  (i-Po 
>  IP  Lkk—i  i  I  . 
k)  Pl  V 1  -  Pi 


^  Fk,k-i  (fc) 

k= 2 i= 1  ' ^ 


A0y  n "■ 


n  e 

77 


EE 

fc=2  i=l 

e(7)  e 


Afe\  1 


fc=2  i=l 

°°  /n2e\2  /A2e 


A0  /  k2 


3= i 


OO/O 

n  e 


j=i 


E(Vllv 


Z=1 

.  \ j 


Ao  y  (*  +  j)2 


Alei  fn 

Ao 


(70) 


In  the  second  inequality,  we  used  the  fact  that  K  is  stochastically  bounded  by  Bin(n,  p)  (see 
Lemma  4).  In  the  third  inequality,  we  used  the  fact  that  po  <  pi  and  i  <  Wk  <  k,  as  well  as  the 
fact  that  n 2  =  o(N),  which  implies  that  pk  ~  {n/N)k .  In  the  fourth  inequality,  we  used  Lemma  19 
and  the  lower  bound  k\  >  ( k/e)k .  The  fifth  inequality  comes  from  a  change  of  variables  and  uses 
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the  definition  of  Wk ■  When  A2e  <  Ao,  since  n2  =  o(N),  this  sum  is  0(n2/N).  When  A2e  >  Ao,  this 
sum  is  equal  to 

■  •4":=log(5)'/nlog(^)  '  (71) 

So  it  suffices  to  show  that  An  — »•  oo.  Since  we  are  working  under  (61),  there  is  c  >  0  such  that, 
eventually, 

ho  log  n  1  -  c 
/A|  log  N  -  1  +  c  ' 

Then,  using  the  fact  that  Ao  V  Ai  =  o(l),  we  have 

/nl°g(^)  =  (1  +  c)^rp(2Ai  -  2/Al  +IXo  -  A0) 

<  —  (1  +  c  +  o(l))  log(n2)  +  (1  —  c)  log  N 

<  log(N/n2)  —  clog  (IV)  , 

eventually.  This  implies  that  An  >  —  1  +  clog N  — >  oo. 

This  concludes  the  proof  of  Theorem  4  under  (61). 


4.1.3  Proof  of  Theorem  4  under  (62) 

Recall  that  p  =  n/(N  —  n)  and  define  ko  =  \bnp] ,  where  b  — >  oo  satisfies  b2(  — >  0.  Let  /cm;n  be  the 
integer  part  of  1  +  j^(l  V  ) .  Define 


n 

Vs  =  P|  {Wt  <  rcfc,  VT  C  S  such  that  \T\  =  k}  , 
k=km[n-\- 1 


where  :=  k  here. 


Lemma  20.  For  any  k  >  km[n  and  any  subset  S  of  size  n,  we  have  Ps[Ts]  =  1  +  o(l). 

This  takes  care  of  the  first  moment.  In  order  to  conclude,  it  suffices  to  control  the  second 
moment,  specifically,  to  prove  that  E[L2]  <  1  +  o(l).  Arguing  as  before,  we  have 


ML2}  < 


E 


l{ir<fc0}exP  (a/l(2)) 


+  E 


l{fe0+i<^<femi„}  exp  (a*<2>) 

+  ®o  H{fc0+i<A'<fcmin}  exp  (2^R/5,in52 


A\  +  Ao  +  A3  . 


2A(9)aV1)  l{„,SinSa<w) 


Arguing  exactly  as  we  did  before,  we  have  A\  =  1  +  o(l). 

Arguing  as  before,  we  also  have 
k  ■ 


A2  <  ^2  exp 

k=ko+ 1 


k  (  A^y^1  -  log 


up 


+  1 


n/mm 

£  E 

k=ko+l 


exp 


k[  l  +  o(l)+  max  <A 

1  AffcoAmin}  ' 


l-l 
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First,  we  have  A(ko  —  l)/2  —  log(koN / n2)  — >  — oo.  This  is  true  if  ko  =  1,  and  when  ko  >  1, 
we  have  N/n 2  <  b ,  so  that 


(Pi  ~P0)2  /M  M  j_2 


6^C->0 


po(l  —  Po)  n462 

by  definition  of  b,  and  therefore 

ko  —  1  N'2  bn2 

We  also  have  A(fcm;n  —  l)/2  —  log(kminN /n2)  — >  —  oo.  To  show  this,  we  divide  the  analysis 
into  two  cases.  When  N 1_a  <  n2_",  this  results  from 


Ah25-h<(l +  „(!)). 


Pl=(l  +  0(l))A:  =  0(l) 


log 


^’m  i  n  A 


(1  —  a^IV1  a  po 

together  with 

2Na 

(1  —  a)na  / 

where  we  used  the  definition  of  fcm;n  and  the  fact  that  Ao  =  ( N/n)a .  When  iV1-a  >  n2~a, 
this  results  from 

r.2  ' 


no 


>  log 


1  —  a 


>  a\og{N/n)  — >  oo  , 


(72) 


^  fcmin  1 


5LMlog(M)+°(1) 


1  ,  2  ,  , 

<  5L— Jlog 

<  iL^Jiog 


1 -al 


o  IV 

'Al 

(1  +  A?)(+ 


n 


2— a 


+  o(l) 

Mi) 


< 


l 


log(l  +  A?)  +  o(l)  +  log  (N/n2 


2^  /  -Mlog(n)  if  a  >  1/3 
—a  log  (N/n)  if  a  <  1/3 


1  —  a 

where  in  the  last  line,  we  have  used  the  identity  [2/(1  —  a)J  =  1  for  a  <  1/3. 
And  we  also  have 

log 


++  )  >  log  (N/n2) 


n * 


(73) 


so  that 


A 


k i  i  \  i  1 


iog(++)  <+- 10 g(i+A?)  4  ° 

\  n2  )  1  —  a  y  —a\og(N/n)  if  a 


>  1/3 
<  1/3  ’ 


which  goes  to  — oo  since  Ai  =  0(1)  and  alog(lV/n)  =  Ao  — >  oo.  Hence,  we  have  A-2  =  o(l). 

It  remains  to  prove  that  A3  =  o(l).  If  we  assume  that  p\  <  2po,  then  A^  <  A  <  po(l  +  o(l)) 
and  we  can  prove  that  A3  =  o(l)  arguing  as  for  A2  above: 


A3  < 


< 


< 


2  exp 
k=km[n-\- 1 
n 

2  exp 
k=km[n-\- 1 
n 

2  exP 

k=km[n-\-l 


7  xk-l  1  k\  \ 
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On  one  hand,  we  have  An  -<  npo  =  ( n/N )1^“  =  o(l).  On  the  other  hand,  log(fcminA/n2)  — > 
oo.  Indeed,  when  iV1^"  <  n2_a,  we  have  (72);  and  when  Nl~a  >  n2~“,  then  N/n 2  > 
n“/(i — »  cx)  and  we  use  (73).  We  conclude  that  A3  =  o(l)  when  p\  <  2po-  In  the  following, 
we  suppose  that  p\  >  2po-  Leaving  unspecified,  so  we  can  use  the  same  arguments  later, 
we  have 


A -A  = 


E0  [l{fc0  +  l<X<fcmin}  eXP 

n  wk 

E  E  P0  [|Si  n  52|  =  fc,  WSlns2  =  i\  exp 


(20W5lnS2  -  2A(9)K^)  1  {wSlns2<WK} 

2 iO  -  2k(2)A(0) 


k — A;min+1  i — 1 
n  wk 

S  E  E 

fc=fcmin+l  *=1 

n  Wk 


n  \  h 
k'P 


z  .Po^-Po)  exP 


2i  log  ^  +  2(&(2)  -  i)  log  f  j — — 
\PoJ  Vi-Po 


:=  £  £^,fc- 

^  —  1 

Furthermore,  since  0  <  1  —  po  <  1  and  1  —  pi  <  1  —  po>  we  have 


Bi,k  < 


n  \  i. 
P 


fc(2) 


2*  <  po(fc)  f 


^(pi/po)21  <  e°W 


\kN 


epi 


(74) 


using  the  standard  bound  (}')  <  ( en/k)k . 

We  now  specify  the  calculations  when  Wk  =  A:.  Considering  the  sums  over  i  =  1, . . .  k/2  and 
over  i  =  A;/2  +  l,...A;  separately,  we  get 


£  Bi,k  <  e 


i=l 


o(fc)  I  en 


2\k 


kN  ) 

2  \  1 
en  \ 


'Lfc/2J 

E 


2—1 


ep\k ^ 
Po 


Ja.  ( ep\k^\ 


Lfc/2J+i 


£  e^J  * 


( ep\k^\  ^  ( ep?fc(2)\ 

+\P^/2J 


-<  e 


o(k) 


f  en 2  \k  (  e3/2n2pi  \  f  e2n2p\  \  k 

{in)  +  J  +  \n^~) 


First,  <  ^  =  o(l)  by  definition  of  k0.  Next,  <  2(p£fo)  ^  =  2y/(  ->  0,  by  the  fact 

that  pi  >  2po-  Finally,  n2p2/ (Npo)  =  A2/Ao  — >  0  since  Ao  — >  oo  and  Ai  =  0(1).  Hence,  we 
conclude  that 

n  k 

£  £>,*  =  0(1).  (75) 

^'=^min  +  l 

This  immediately  implies  that  A3  =  o(l). 

This  concludes  the  proof  of  Theorem  4  under  (62). 

Proof  of  Lemma  20.  Let  us  consider  the  event 


rs  :=  {no  connected  component  of  Qs  has  more  than  one  loop} 
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Under  Tg,  a  connected  component  of  Qg  has  at  most  as  many  edges  as  vertices.  Consequently, 
Ts  C  Us,  and  we  only  need  to  prove  that  Ps(rg)  =  1  +  o(l).  Since  limsupAi  <  1  and  IPsfT^)  is 
nondecreasing  in  Ai,  we  may  assume  that  Ai  is  fixed  in  (0, 1). 

As  a  warmup  for  what  follows,  we  note  that  the  number  L&  of  loops  of  size  k  in  Qs  satisfies 


E5[L  k\=v\ 


n\ 


<f! 

(n  —  k)\2k  2k 


since  there  are  n!/[(n  —  k)\2k]  potential  loops  of  size  k.  Now,  if  a  connected  component  contains 
(at  least)  two  loops,  there  are  two  possibilities: 


•  The  two  loops  have  at  least  one  edge  in  common.  In  that  case,  there  is  a  loop  (say  of  length 
k )  with  a  chord  (say  of  length  s  <  k).  Let  L(,  s  denote  the  number  of  such  configurations, 
There  are  n!/[(n  —  k)\2k]  potential  loops  of  size  k.  Given  a  loop  of  size  k,  there  are  less  than 
(2)  starting  and  ending  nodes  possible  for  the  chord.  Once  these  two  nodes  are  set,  there 
remains  less  than  n!/(n  —  s  +  1)!  possibilities  for  the  other  nodes  on  the  chord.  Thus,  we  have 


MK,s}<pi 


k+s 


n\ 


n\ 


k-\-s 


.k 


<  (  —  )  knk+s~l  <  \\+s-  . 

n 


(n  —  k)\2k\2J  (n  —  s  +  1)! 

Summing  this  inequality  over  s  and  k,  we  control  the  expected  number  of  loops  with  a  chord: 


00  k—  1 


OO 


E 


k\k+1 

1  —  Ai 


since  limsupAi  <  1.  Hence,  this  event  occurs  with  probability  going  to  0. 


•  The  two  loops  have  no  edge  in  common.  Since  there  are  in  the  same  connected  component, 
there  is  a  path  that  goes  from  a  vertex  in  the  first  loop  to  a  vertex  in  the  second  loop.  Let 
us  note  L'fci  k2  s  the  number  of  loops  of  size  k\  and  k2  that  do  not  share  an  edge  and  are 
connected  by  a  path  of  length  s.  Observe  that  there  are  less  yn_^y2kl  possible  configurations 
for  the  first  loop,  less  than  ^.n_^y 2k2  possible  configurations  for  the  second  loop,  and  less  than 
kik2n\/{n  —  s  +  1)!  possibilities  for  the  chord.  Thus,  we  get 

+s  n\  n\  n\ 

(n  —  k\)\2k\  (n  —  k2)\2k2  1  2  (n  —  s  +  1)! 

ki+k2+s  \ki+k2+s 

k\+k2+s-l  _  m _ 

n  —  , 

Tl 

so  that  the  expected  number  of  such  configurations  is  bounded  as  follows 

E  E E E [LUJ <(EEE T+‘i+* <-(1) ■ 

fci>3fc2>3s>l  fci>3  fc2>3  s>l 

Hence,  this  second  event  occurs  with  probability  going  two  zero 
All  in  all,  we  have  proved  that  Ps(r'5)  =  1  +  o(l),  implying  that  P^Ts)  =  1  +  o(l).  □ 


E  [Lfci,fc2,s]  <  Pi 


ki  + 


< 

n 
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4.1.4  Proof  of  Theorem  4  under  (63) 


We  follow  the  arguments  laid  out  for  the  case  (62).  We  define  Tg  in  the  same  way,  except  that 
wk  ■=  [k{1f-a~]  >  where  c  is  a  positive  constant  (to  be  chosen  small  later)  such  that  c  <  a  and, 
eventually, 

sup  ^E s[Wk,s]  <  \z~f~  ■  (76) 

n/u]y<k<n  ±  Qi 

Lemma  21.  For  any  k  >  fcm;n  and  any  subset  S  of  size  n,  we  have  IPsfTs]  =  1  +  o(l). 

For  the  second  moment,  we  proceed  exactly  as  in  the  case  (62),  and  we  start  from  (75).  In 
fact,  when  Wk  <  k,  the  proof  is  complete.  So  we  assume  that  c  is  small  enough  that  Wk  >  k,  and 
bound  the  sum  over  k  +  1  <  i  <  Wk-  For  i  >  k,  we  use  the  bound  (74),  together  with  the  fact  that 
Ao  =  ( N/n)a  and  k  <  i,  to  derive 

2  \  k 
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f  en2 
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This  allows  us  to  control  the  sum 
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n  \  k-(i-a)wk  /  Afe 


V  1 


Wk 
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o(k)+k  (  n  \  fc(1_(1_c)1/2)  (  Afe 

te<>  y  tvi 


.(1-4 


1/2 


=  exp 


0(k )  —  k(  1  —  (1  —  c)1//2)  log(iV/n) 


where  in  the  second  line  we  used  the  fact  that  Wk  =  O(k)  since  limsupa  <  1,  and  in  the  third  line 
we  used  the  fact  that  Ai  =  0(1).  Thus, 

n  wk 

Y  Y  Bhk  =  °(1)  > 

k= i=k-\- 1 

which  together  with  (75)  allows  us  to  conclude  that  A%  =  o(l). 

This  concludes  the  proof  of  Theorem  4  under  (63). 

Proof  of  Lemma  21.  Recall  that  un  =  loglog(iV/n).  First  we  consider  integers  k  satisfying  km\n  + 
1  <  k  <  n/uN-  Define  u'k  =  k(  1  —  c)-1/2  V  1^  and  q'k  =  uj'k/k^2\  Applying  a  union  bound  and 
Chernoff’s  bound  for  the  binomial  distribution,  we  derive  that 


P  s[Wls>Jk]  < 


n 


k 

<  exp 


P[Bin(fc^2\pi)  >  u)k] 
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Since  k/n  <  1/un  =  o(l),  and  since  Ai  is  bounded,  we  have  q'k/pi  — >  oo,  so  that 


k  -  1 


Ai 


V  1 


n 


log|^Tj+log|(1-c,'1/2(1VS 


=  (i  -  c)-1/2 
>  (l  +  o(l))(l  -c)_1/2log(5)  > 
and  therefore,  since  c  E  (0, 1)  is  fixed, 

log  (y )  -  <  1  +[!-(!  +  o(l))(l  -  cr1/2]  log(nTv)  ->  -oo  . 

We  conclude  that 

n/uff 

E  ps[^,s^y  =o(i)  • 

A;=/cmin_l_  1 

Let  us  now  prove  that  uk  <  wk.  Indeed,  this  inequality  holds  if,  and  only  if,  Ai  <  2(1  —  c)/(l  —  a) 
and  c  <  a.  The  second  inequality  is  by  definition  of  c,  while  the  first  inequality  is  ensured  by  (76) 
since 

Al  U  1  =  E s[W*tS/n]  <  supE s[Wls/k]  <  (1  -  2c)/ (1  -  a)  . 


2  n 


k<n 


Let  us  turn  to  integers  k  satisfying  k  >  ti/un ■  Let  co  =  (1  —  c)  1!2  —  \  and  t  =  cq  Es[W£s].  By 
taking  any  fixed  subset  T  C  S  of  size  \T\  =  k,  we  derive 


E sms]  >  MWt]  =  PikW  >  ^{n/uN)W 
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u 


oo  , 


(77) 


TV 


so  that  t  satisfies  the  condition  of  Lemma  7  eventually.  Using  that  lemma,  we  derive  that 
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By  Condition  (76),  wk  >  EgfW/ s](l  —  c)  1//2.  Hence,  there  exists  a  positive  constant  k,  such  that 


E  e5  [wis  >  Wk\  <  y,  exp[-KEs[^i)S]] 

k=n/uN  k=n/uN 


<  nexp  — kEs 

Because  of  (77)  and  the  fact  that  log (IV)  =  o(n),  we  have 
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and  therefore  the  sum  above  goes  to  0. 
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4.2  No  test  is  asymptotically  powerful 

When  Ao  is  bounded  away  from  0  and  infinity,  the  triangle  test  has  some  non-negligible  power 
as  long  as  Ai  is  bounded  away  from  0  (see  Section  3.4).  This  motivates  us  to  obtain  sufficient 
conditions  under  no  test  is  asymptotically  powerful. 

Our  method  is  also  based  on  bounding  the  first  two  moments  of  a  truncated  likelihood  ratio  L. 
Indeed,  it  is  enough  to  show  that  liminfEo-L  >  0  and  liminfEo[T2]  <  oo.  This  comes  from  the 
following  result. 

Lemma  22.  Let  Po  and  Pi  be  two  probability  distributions  on  the  same  probability  space,  with 
densities  fo  and  f\  with  respect  to  some  dominating  measure.  Let  T  be  any  event  and  define  the 
truncated  likelihood  ratio  L  =  L  Hr,  where  L  =  f\/ /o  is  the  likelihood  ratio  for  testing  Po  versus  Pi. 
Then  any  test  for  Pq  versus  Pi  has  risk  at  least 


4  (E0L)3 
27e0[L2]  ’ 

where  Eo  denotes  the  expectation  under  Po,  and  by  convention  0/0  =  0. 

Proof.  Assume  Eo  L  0,  for  otherwise  the  result  is  immediate.  The  risk  of  the  likelihood  ratio  test 
{L  >  1}  —  which  is  the  test  that  optimizes  the  risk  —  is  equal  to 

B  :=  1  -  *  E0  \L  -  1|  =  1  -  E0(l  -  L)+ >  1  -  E0(l  -  L)+  , 

since  L  <  L.  For  any  t  G  (0, 1),  we  have 

E0(l  —  L)_|_  <  (1  -  t)P0(L  >  t)  +P0(L  <t)  =  1  -  tP0(L  >  t)  . 

Moreover,  using  the  Cauchy-Schwarz  inequality,  we  have  for  any  t  >  0 

E0L  =  E0[Ll|^<t|]  +E0[Ll|£>tj] 

<  t  +  ^E0[L2]  P0(L  >  t)  , 


so  that,  taking  t  <  Eq  L,  we  have 


Po(L  >  t)  > 


(Eo  L-t)2 
Eo  L2 


We  conclude  that 

B  >  t¥0(L  >t)>t  (E°Zr*)2  , 

E0  L2 

and  optimizing  this  over  0  <  t  <  Eq  L  yields  the  result. 


□ 


Since  we  only  need  to  focus  on  the  case  where  Ao  is  bounded  from  0  and  infinity,  and  where  Ai 
is  bounded  from  0  (because  the  other  cases  are  covered  by  Theorem  4),  we  may  assume  they  are 
fixed  without  loss  of  generality.  In  that  case  f  ►  0  is  equivalent  to  n2 /N  — >  0,  which  is  what  we 
assume  in  the  following. 
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Theorem  5.  Write  n  =  NK  with  0  <  k  <  1/2,  and  assume  that  Ao  and  Ai  are  both  fixed.  No  test 
is  asymptotically  powerful  in  all  the  following  situations: 

Ai  <  1,  A?e  <  A0  ;  (78) 

Ai  <  1,  A?e>A0,  1~2K  >1-  (79) 

K  1os[jw) 

Proof  of  Theorem  5.  We  use  the  same  truncation  as  in  Section  4.1.2,  and  still  denote  the  resulting 
truncated  likelihood  by  L. 

For  the  first  moment,  by  symmetry, 

E0[L]  =  Fs[Ts]  =  Ps[£s  is  a  forest,  |CmaXis|  <  fn]  ■ 

We  already  saw  that  Ps[|Cmax,s|  </«]->  1  (Van  der  Hofstad,  2012,  Th.  4.4).  Consequently, 

E0[Z]  =  ¥s[Gs  is  a  forest]  +  o(l)  . 


Of  course,  Qs  is  a  forest  if,  and  only  if,  it  has  no  cycles.  By  Takacs  (1988),  the  number  of  cycles  in 
Qs  converges  weakly  to  a  Poisson  distribution  with  mean 


o(Ai) 


1  —  Ai 


Ai  _  A? 

2  4 


when  Ai  <  1  is  fixed.  As  a  consequence,  Eo[T]  =  exp  [— a(Ai)]  +  o(l),  which  remains  bounded  away 
from  zero. 

For  the  second  moment,  we  start  from  (70): 


Eq 


with  fn  =  (1  +  c)IAi1logn  and  c  is  a  small  positive  constant.  Under  (78),  we  have  Afe  <  Ao  and 
the  RHS  is  0(n2/N)  =  o(l).  Under  (79),  we  have  \\e  >  Ao,  and  the  RHS  is,  as  before,  equal  to 
(71).  Here  we  have 

An=  [l  -  2k  -  (1  +  c)-^-log  ]  logiV  oo  , 

when  (79)  is  satisfied  and  c  is  small  enough.  Hence,  in  any  case,  we  found  that  Eo  [L2]  <  1  +  o(l). 

□ 


5  Some  tests  running  in  polynomial  time 

There  is  a  surge  of  interest  in  decision  theory  with  computational  constraints,  the  main  question 
being,  what  can  be  achieved  with  tests  that  can  be  computed  in  polynomial-time.  An  example  is 
the  planted  clique  problem  mentioned  in  the  Introduction,  where  it  is  conjectured  that  there  are  no 
polynomial-time  algorithms  that  can  detect  with  high  confidence  the  presence  of  a  clique  of  size  k 
added  to  Q  ~  G(JV,  1/2). 

To  focus  the  discussion,  assume  that  the  total  degree  test  (which  runs  in  0(N2)  time  at  most) 
is  not  asymptotically  powerful.  In  fact,  to  be  even  more  specific,  assume  that  limsup  Ai  <  oo  and 
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that  n2 /N  =  o(l).  In  this  setting,  the  largest  connected  component  test  —  which  can  be  computed 
in  0{\£\  + 1  V|)  =  0(N 2)  time  —  is  powerful  when  lim  inf  Ao  <  1  and  Ai  is  sufficiently  large.  And  the 
triangle  test  —  which  can  be  computed  in  0(N 2’82)  time  (Alon  et  ah,  1997)  —  is  not  asymptotically 
powerful  unless  Ao  — >  0.  Those  are  the  only  tests  that  we  studied  here  which  we  can  compute  in 
polynomial  time.  The  k- tree  test  is  powerful  when  Ao  is  sufficient  small  and  Ai  is  sufficiently  large, 
but  we  do  not  know  how  to  compute  it  in  polynomial-time.  And,  similarly,  the  broad  scan  test  is 
asymptotically  optimal  when  Xq  >  e,  but  we  do  not  know  of  any  polynomial-time  algorithm  for 
computing  it. 

In  this  section,  we  discuss  a  few  other  tests  that  can  be  computed  in  polynomial  time.  Here 
is  a  brief  summary  of  the  best  detection  bounds  and  the  corresponding  tests.  In  the  poissonian 
asymptotic  where  Ao  and  Ai  are  both  fixed,  a  polynomial  time  approximation  of  the  number  of 
fc-cycles  (with  k  large)  is  powerful  as  soon  as  Ai  >  \/Ao  VI.  In  the  asymptotic  Ao  — >  oo  with 
log (IV)  -<  Ao  A  nlog(IV),  the  test  based  on  the  second  eigenvalue  of  W  is  powerful  for  Ai  -y/Ao. 
For  nlog(IV)  -<  Ao  A  N/n,  a  convex  relaxation  of  the  n-sparse  eigenvalue  problem  is  powerful  for 

A! »  v^mjv)  v  (a„  log(jv))1/-  v  yv  1/J. 


5.1  The  maximal  degree  test 

The  test  based  on  the  maximum  degree  was  examined  in  (Arias-Castro  and  Verzelen,  2012)  un¬ 
der  the  assumption  that  Ao  log  IV,  and  was  shown  to  be  asymptotically  powerful  when  Ai 
y/XfiTogN.  We  now  concentrate  on  the  case  where  Ao  -<  log N. 

Recall  the  function  h(x)  =  xlogx  —  x  +  1  that  appears  in  Lemma  2.  Below,  /i_1  refers  to  the 
inverse  of  h  :  [1,  oo)  — >  [0,  oo),  where  it  is  strictly  increasing.  We  will  also  use  the  fact  that 


h(ab)  =  ah(b)  +  bh(a)  +  (o  —  1)(6  —  1)  ,  Va,  b  >  0  .  (80) 

Proposition  5.  Assume  that  n  =  NK  with  k  £  (0,1/2)  fixed,  and  that  Ao  =  70  log  N  and  X\  = 
71  log  N,  where  70  V  71  =  0(1)  and  —  log(7o  A  71)  =  o(loglV).  Then  the  maximum  degree  test  is 
asymptotically  powerful  if 

lim  inf  £  >  1  ,  g  :=  (l  +  —k)  w  iT  ’ 
and  powerless  if  lim  sup  g  <  1.  In  particular,  when  70  — >  0,  the  test  is  powerful  if 


lim  inf  >  1  , 


«log(l/7o) 

log(l/7i) 


and  powerless  if  lim  sup  <  1. 


The  proposition  implies  that  the  test  is  powerless  when  Ao  and  Ai  are  both  fixed.  The  regime 
where  Ao  =  0(N~a° )  for  some  ao  >  0  fixed  is  excluded.  However,  in  this  very  sparse  regime,  the 
maximum  degree  converges  to  a  constant  —  except  for  special  circumstances,  see  (Bollobas,  2001, 
Th.  3.2)  —  and  the  statement  of  Proposition  5  would  have  to  be  modified  with  careful  rounding. 


Proof  of  Proposition  5.  Let  A  (PC)  denote  the  maximum  degree  of  a  graph  TL.  Below  A  is  a  short¬ 
hand  for  A(C7).  We  first  prove  that,  in  probability  under  Po> 

A 

7o/i“1(l/7o)logIV 

Let  po  =  Ao /N  and  ??  =  h-1(log(IV)/Ao). 


(81) 


Upper  bound.  First,  let  k  =  |~(1  +  £)??Ao~|  •  The  expected  number  of  nodes  with  degree  at  least 
k  is  equal  to 

N P  [Bin(JV  -  l,p0)  >  k ]  <  N ex.p[-NHPo(k/N)] 

<  exp  [log  AT  -  X0h((l  +  s)r])  +  0(\lrj2/N)\  , 

using  Lemma  1  and  then  Lemma  2,  with  the  fact  that  HPo(k/N )  >  HPo((  1  +  e)rjpo).  Note 
that  Aq rj2 /N  =  0(log(N)/N)  =  o(l)  in  our  context.  We  then  use  (80),  to  get 

log N  —  Aoh((l  +  e)rj)  <  — e log N  — >  — oo  . 

We  conclude  with  Markov  inequality  that  Po(A  >  k)  — >  0. 

Lower  bound.  For  the  lower  bound,  redefine  k  =  [(1  —  s)r]\o\.  We  could  use  (Bollobas,  2001, 
Th.  3.2),  but  we  will  derive  the  result  ‘by  hand’,  because  the  same  argument  is  used  below. 
Let  Dk  denote  the  number  of  nodes  with  degree  exactly  k,  so  that  Dk  =  Yli  ^{Wi=k}i  where 
Wi  is  the  degree  of  node  i.  Note  that  Wj  ~  Bin(iV  —  l,po)  under  Pq.  Using  Stirling’s  formula, 
we  have 

E0[L>fc]  =  JVP[Bin(JV-l,po)  =  fc] 

x  exp  [log IV-  log k  (N  -  l)Hpo(k/(N  -  1))] 

>  exp  [loglV  —  ^  log  k  —  A0h((l  —  e)rj)  +  0(X^rf' /m)\  , 

applying  Lemma  2  in  the  third  line.  We  note  that  X^rj2  /N  =  o(l)  as  before,  and 

^’|A°°0(log(logW/Ao)))=0(1°gAr)- 

so  that  logfc  =  O(loglogJV).  Applying  (80),  we  then  get 

log N  —  X0h((l  -  £)ri)  >  [e  -  jj- yh(l  -  e)]  log  N  . 

Letting  c  =  liminf  log(lV)/Ao  >  0,  we  have  p/h(rf)  <  h~1(c)/c,  and  e  —  j^h(  1  —  e)  > 

e  —  h  1  —  e)  >  0  for  e  >  0  small  enough.  We  therefore  have  that  Eo[-Dfc]  — ; ►  oo.  We  now 

turn  to  the  variance.  Let  qk  =  P[Bin(iV  —  2,po)  =  k].  For  i,j  6  V  distinct,  conditioning  on 
Wij,  we  get 

P0[Wg  =  k,  Wj  =  k]  =  (1  -  Po)q2k  +  Poql-i  , 

while 

P0[fFi  =  k\  =  (  1  -  Po)qk  +  PoQk-i  , 

obtaining 

Cov0(l {w.=k},t{Wj=k})  =  Po(l  -  Po)(Qk  ~  Qk- 1)2  • 


Var 0[Dk]  <  E0[L>fc]  +N(N  -  l)p0(l  ~po)(qk  ~  qk- 1)2  , 


Eventually, 


41 


so  that 


Varo[£>fc] 

(EoPfc])2 


< 


1 

IEo[Ac] 


1 

Eopf] 

1 

MDsk] 


N2po(qk  ~  qk- 1)2 
N2((i  -  po)qk  +  poqk-i)2 

+  O(p0)(i  +  -M-) 

% 

+  °(po)  +  0(-^)  ^  o  , 


since  Po^fr~  ~  p^N2  ~<  ^°\Jn  — >•  0.  Hence,  by  Chebyshev  inequality,  Dk  — >  oo  under  Po, 
implying  that  Pq(A  >  k)  — >•  1. 


The  proof  of  (81)  is  now  complete. 

Henceforth,  we  work  under  the  alternative  Pg.  Define  A s  '■=  maxjgsH7,.  We  have  Wi  = 
Wf  +  Wfc ,  where  Hf  =  £jeT  Wjj  for  T  C  V.  (Recall  that  Wa  =  0.)  Our  arguments  are  parallel 
to  those  we  used  under  Po,  the  only  difficulty  being  that  W{  is  not  binomial  anymore.  Indeed, 
Wf  Bin(n  —  l,pi)  and  Wf  ~  Bin(A  —  n,po)  are  independent.  Nevertheless,  the  resulting 
Poisson  binomial  distribution  is  close  to  a  binomial  distribution. 


Lemma  23.  Suppose  X\  ~  Bin(mi,  q\)  and  X2  ~  Bin(m.2,  92)  are  independent.  Let  m  =  mi  +  m2 
and  q  =  •  Then,  for  k  >  mq, 

P[Xi  +  X2  >  k]  <  P [Bin{m,  q)  >  k\  . 


If  qi  V  q2  <  1/2  and  k  <  (mi  A  m2)/2,  we  also  have 


exp 


— qk  — 


(mq)2  +  k 2 
m±  A  m2 


< 


P[Xi  +  A2  =  k] 


P  [Bin(m,  q)  =  k] 


<  exp 


(qi  V  q2)k  + 


(mq)2  +  k 
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Equipped  with  Lemma  23,  we  proceed  with  controlling  A 5  under  P5  using  the  same  arguments. 
In  fact,  we  prove  that,  under  P 5, 


A, 


(7i«  +  7o )h 


1  . 


log  A 


(82) 


•  Upper  bound.  Let  p  =  (n  n'^Po  and  A  =  (N  —  1  )p.  For  e  >  0  small,  consider 

k  =  (1  +  s)A/i_1(log(n)/A).  As  before,  k  =  O (log  IV),  and  we  have 


Ps  [As  >  k]  < 
< 


< 


nP  [Bin(A  —  l,p)  >  k] 
nexp[-(N-l)Hp(k/(N-l))] 
exp  [logn  —  A  h(k/ A)  +  0(k2/N)~\ 
exp  [  —  elogn  +  0((log  A)2/A)]  — »•  0  , 


The  first  line  comes  from  the  union  bound  and  Lemma  23,  the  second  line  from  Lemma  1, 
the  third  from  Lemma  2,  and  the  fourth  from  (80).  Now,  since  A  ~  Ai  +  Ao,  it  is  easy  to  see 
that 


A h  1(\og(n)/\)  ~  (Ai  +  A 0)h  1(log(n)/(Ai  +  A0))  ~ 


(71^  +  70 )h  x( 


K 

7o  +  7i« 


) 


log  A  . 
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We  conclude  that,  in  probability  under  P 5, 


lim  sup  y 


A  5 


(71K  +  70 )h  x| 


70+71 


n) 


<  1 . 


log  N 


Lower  bound.  Let  denote  the  number  of  nodes  in  S  with  degree  equal  to  k.  Obviously, 
A  >  k  if  Dj!  >  1.  Fix  e  small,  and  redefine  k  =  [(1  —  e)\h  1 ( log(n)/A)J .  We  have 


Es,[Df]  =  n¥s[Wi  =  k}  >  ne 


-pk-¥+£ 


in(lV  —  l,p)  =  k] 

>-  ^exp  [-  (N-  l)Hp(k/(N-  1))] 

=  exp  [logn  —  ^  log  k  —  A  h(k/\)  +  0(k2/N)]  — >  00  , 

where  we  used  Lemma  23  in  the  first  inequality;  in  the  second  the  fact  that  pk+  \2-i  =  o(l) 
since  k  =  O(logiV),  A  =  0(log IV),  p  <  p\  =  0(log(N)/n)  and  n  =  NK,  as  well  as  Stirling’s 
inequality;  we  then  applied  Lemma  2.  The  divergence  to  infinity  follows  from  the  same 
arguments  that  we  used  before.  We  now  bound  the  variance  of  D^.  Redefine  q k  =  P[A  +  Y  = 
k]  where  X  ~  Bin(n  —  2,pi)  and  Y  ~  Bin(A  —  n,po )  are  independent.  Working  exactly  as 
we  did  before,  we  get 


Va.r,s[Pf']  ^  1  ,  /  gj-ix 

(EsI^f])2  “  Es[D%]  +PL[  +  q2  ’  ' 


Let  qk  =  P[Bin(iV  —  2 ,p)  =  k ],  where  p  :=  '^Po,  and  also  A  =  (N  —  2 )p.  By 

Lemma  23, 

qke  "-1  <  qk  <  Qk  e  JV'1  , 

so  that  qk  ~  qk,  and  therefore 

Qk- 1  1  k2  Ai  (log  A)2 

Pl~T~  ~  Pl—PT  ~  Pi 77777  ^  77 - 7 -  =  Of1)  • 


p2N2  Aq 


n 


Hence,  Nk  -+  00  under  P5,  by  Chebyshev  inequality.  This  implies  that  A g  >  k  with  prob¬ 
ability  tending  to  one  under  P 5.  Then,  arguing  as  for  the  upper  bound,  we  conclude  that 
under  P 5 

As 


lim  inf 


>  1 


(71^  +  70  )h  1{^^)\\ogN 

So  we  proved  (82),  and  together  with  (81),  we  can  conclude. 


□ 


Proof  of  Lemma  23.  The  upper  bound  is  a  special  case  of  (Hoeffding,  1956,  Th.  4).  Letting  A j  = 
mjqj  and  A  =  mq,  the  lower  bound  goes  as  follows: 

P[A"i  +  X2  =  k]  =  IP[Bin(mi,  gi)  =  &q]  P[Bin(m2,  qi)  =  k2] 

k\+k2=k 


> 


£  «- 
ki+k2=k 


A  f+fcf 


,-Ai 


A^ 

h\ 


^re-A2^ 

k2\ 


A  2+fc2  Afc 

>  g  m1Am2  g  A - 

k\ 

_  7  _ 

>  e  9  rn1  Am2  P[Bin(m,  g)  =  fc]  , 


using  (Bollobas,  2001,  Eq.  1.14)  in  the  second  line  and  (Bollobas,  2001,  Eq.  1.13)  in  the  fourth  line. 
The  upper  bound  is  obtained  similarly.  □ 


5.2  A  relaxation  of  the  n-sparse  eigenvalue  problem 

We  also  studied  in  (Arias-Castro  and  Verzelen,  2012)  a  test  based  on  a  convex  relaxation  of  the  n- 
sparse  eigenvalue  problem.  Formally,  for  a  positive  semidefinite  matrix  B  <E  RNxN  and  1  <  n  <  N, 
define 

/3nax(B)  =  max  ||BS||  , 

\S\=n 

where  B5  denotes  the  principal  submatrix  of  B  indexed  by  S  C  {1, . . . ,  IV}  and  ||B||  the  largest 
eigenvalue  of  B.  d’Aspremont  et  al.  (2007)  relaxed  this  to 

SDPn(B)  =  max  trace(BZ),  subject  to  Z  y  0,  trace(Z)  =  1,  |Z|i  <  n  , 
z 

where  the  maximum  is  over  positive  semidefinite  matrices  Z  =  (Zst)  G  RNxN  and  |Z|i  =  JV  f  \Zst\. 
We  considered  in  (Arias-Castro  and  Verzelen,  2012)  the  relaxed  scan  test,  which  rejects  for  large 
values  of  SDP„(W2).  With  the  same  method  of  proof  that  we  used  there,  we  can  conclude  that  the 
test  is  asymptotically  powerful  when  n  =  NK  with  0  <  k  <  1/2  fixed,  Ao  <S  and 

Ai  >  y/nlog(N)  \/  (A0log {N))1/l\J  ^ 


5.3  The  number  of  fc-cycles 

Let  Ck  denote  the  number  of  simple  fc-cycles  in  the  graph.  To  test  against  a  stochastic  block  model 
alternative,  Mossel  et  al.  (2012)  use  the  test  based  on  Ck,  with  k  — >  00  slowly.  Our  arguments  are 
also  based  on  the  first  two  moments  of  Ck,  but  controlling  the  variance  is  more  involved. 

Proposition  6.  The  test  based  on  Ck,  with  k  — >  00  and  k  =  0(log  N)1^,  is  asymptotically  powerful 
when  Aq  and  Ai  are  fixed  with  X\  >  \/Aq  V  1. 


Although  computing  the  number  of  simple  fc-cycles  seems  difficult,  Alon  and  Gutner  (2010) 
provide  an  approximation  that  suffices  for  our  purposes.  They  show  that,  for  any  <5  G  (1,2),  there 
is  a  (deterministic)  function  Ck  that  can  be  computed  in  less  than  g(k,  5) N3  log  N  time  on  graphs 
with  N  nodes,  such  that 


sr— l 


< 


Ck(H) 

ckm 


<  5 


for  any  fixed  graph  %.  In  fact,  g(k)  =  e°(klos^gk  fciog(i  <5))_ 


Proposition  7.  Take  6  =  1  +  exp[— log2(log(A))]  (for  example).  The  test  based  on  Ck,  with 
k  — '?  oc  and  k  =  0(loglog  N)  runs  in  polynomial  time  and  is  asymptotically  powerful  when  Ao  and 
Ai  are  fixed  with  Ai  >  \/Ao  V  1. 


Proof  of  Proposition  6.  We  aim  at  applying  (47).  Straightforward  calculations  show  that 
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since  there  are  (N_^ky2k  possible  cycles  with  k  edges,  and  k 2  =  o(N).  For  the  variance,  arguing 
as  in  (Bollobas,  2001,  Eq.  4. 2, 4. 4),  which  is  based  on  bounding  the  number  of  pairs  of  cycles  that 
share  at  least  one  node,  we  get 


MCk{ck  - 1)]  < 


< 

< 


N\p\ 


2k 

’0 


(N  -2k)\{2k)2 
/  Nlpk 


2k— 1 

+  E 


l=k 


A  k\ 


(N-k)\(2k) 


)  +P6 


l/k 


(2  ky 


k J  2k 

2k— 1 

sr  \t_ 

tk 


Po 

l\ 


(Eo[Cfc])2  +  A^-1/fc(A0Vl)2fc(2fc)! 


Hence, 

Var0[Cfc]  =  Eo[Cjfc]  +  E0 [Ck(Ck  -  1)]  -  (E0[Cy)2 
<  E0[Ck}  +  N~1/k(X0Wl)2k(2k)\  , 


with 


JV_1/fc(A0  V  l)2k(2k)\  =  exp 


—  -  log(IV)  +  0(k\ogk ) 
k 


since  k  =  0(log  TV)1/4.  Hence,  Varo[Cfc]  <  Eo[Cfc]  +  o(l).  By  Chebyshev’s  inequality,  it  follows 
that,  under  Po, 

Ck  -  E0[Ck }  <  o(l)  +  O(Xk0/k)1/2  .  (84) 

Let  us  turn  to  the  alternative  hypothesis.  Let  C f  refer  to  the  number  of  fe-cycles  whose  nodes 
are  in  S.  We  use  the  decomposition  Ck  =  Ck  +  (Ck  —  C^).  Note  that  Eo[C^]  <  (Ao n/N)k  =  o(l), 
so  that  Ck  =  o(l)  under  Po  by  Markov  inequality.  Hence,  since  Ck  —  C is  stochastically  larger 
under  P5,  it  follows  from  (84)  and  that,  under  P5, 


Ck  -  Ci  >  E0[Ck]  -  o(l)  -  Q{Xk/k)1/2  . 


On  the  other  hand,  arguing  as  before,  we  derive  that  E^  ~  ^ 


and,  under  P5, 


Ci>Es[cS]-o(l)-0{Xk/k)l/2  . 


Together,  we  hnd  that,  under  P5, 

Ck  -  Eo[Ck]  >  (1  +  o(l))|  -  o(l)  -  0(Xk/k)1/2  -  0(Xk/k)1/2  .  (85) 

since  Ai  >  1.  Comparing  the  control  under  the  null  (84)  with  the  control  under  the  alternative  (85), 
and  using  the  fact  that  Ai  >  yAo,  allows  us  to  conclude  that  the  test  rejecting  for  Ck  >  Eo[Cfc]  +  A 
is  asymptotically  powerful.  O 


Proof  of  Proposition  7.  For  5  =  1  +  exp[— log2(log(lV))]  and  k  =  O(logloglV), 

g(k)  =  exp  [0(log3  log(lV))]  =  fVo(1)  , 

and  the  test  runs  therefore  in  polynomial  time.  In  order  to  analyze  the  power  of  the  test,  we  use 
Chebyshev’s  inequality  and  follow  the  tracks  of  Proposition  6.  Under  the  null  hypothesis, 


Ck 


Ck<(S-l)Ck  < 


(6-  1) 


Eo[cy  +  Op0 


VMCk] 


+  oPol1) 


op0(l)  , 


since  (5  —  l)Eo[Cfc]  ~  e  log2 log(A^) ^  _  G(i)j  as  k  =  O (log log (IV)).  Under  the  alternative  hy¬ 
pothesis,  we  derive  similarly  that  Ck  >  5~1Ck  >  Ck  —  ops(l).  Thus,  we  can  argue  as  in  proof  of 
Proposition  6.  □ 
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5.4  Spectral  methods 

‘Spectral  methods’  are  procedures  that  rely  heavily  on  an  eigen-decomposition  of  the  adjacency 
matrix,  or  related  matrices  like  the  (normalized)  graph  Laplacian.  Since  the  adjacency  matrix  can 
be  recovered  from  its  spectral  decomposition,  any  method  is  in  principle  spectral,  but  the  term  is 
reserved  for  methods  that  are  computationally  tractable  (Alon,  1998;  Arsic  et  al.,  2012;  Chung, 
1997;  Pothen  et  ah,  1990).  We  explore  such  methods  here.  Let  (3i{TL )  >  •  ■  ■  >  (3n(TL)  denote  the 
eigenvalues  of  a  graph  TL  on  N  nodes,  meaning  the  eigenvalues  of  its  adjacency  matrix.  Let  (3k  be 
short  for  (3k(G)- 

We  first  prove  a  simple  result  for  the  test  that  rejects  for  large  values  of  (3\. 

Proposition  8.  The  test  that  rejects  for  large  values  of  (3\  is  asymptotically  powerful  when 


Ai  >  A0  > 


/  log  A" 
log  log  N 


Proof.  Krivelevich  and  Sudakov  (2003)  showed  that,  for  Q  ~  G(m,  A/m),  (3\{G)  ~  y 1  A(G)  V  A  in 
probability  when  the  RHS  goes  to  infinity,  and  when  A2  3$>  log(m)/ loglog(m),  A  yj A(G)  by 
Proposition  5  and  the  fact  that  h~1(x)  ~  x/logx  when  x  —>  oo. 

Hence,  under  Po>  Pi  ~  Ao,  while  under  Pg,  (3\  >  (3\{Gs)  ~  Ai.  Based  on  that,  we  conclude.  □ 

Based  on  the  result  of  Krivelevich  and  Sudakov  (2003),  and  also  the  fact  that  (3\  >  2(t" ,  we  would 
intuit  that  the  test  based  on  (3\  would  behave  as  the  total  degree  test.  However,  the  deviations  of 
[3 1  are  rather  intricate  (Janson,  2005).  A  similar  fine  analysis  would  have  to  be  obtained  under  the 
alternative  to  really  understand  the  power  properties  of  this  test. 

The  test  based  on  (32  is  particularly  attractive  within  spectral  methods  because  of  its  role  in 
clustering.  We  obtain  the  following  asymptotic  performance. 


Proposition  9.  The  test  that  rejects  for  large  values  of  (32  is  asymptotically  powerful  when  Ao  >~ 
logA  and  Ai  »  -y/Ao  V  (A0-y). 

Note  that  Ai  3$>  Ao^  is  equivalent  to  p\  i$>  po,  which  is  for  example  true  when  limsupa  <  1. 

Proof  of  Proposition  9.  We  start  by  controlling  (32  under  the  null.  For  this,  we  use  the  following 
result. 

Lemma  24.  When  A  >-  log m,  A/m))  =  0(V A). 

Proof.  When  A  >-  (logm)6,  the  proof  of  Fiiredi  and  Komlos  (1981)  works  in  exactly  the  same  way, 
to  show  that  (32(G(m,  \/m))  =  0(V A).  This  was  observed  by  Feige  and  Ofek  (2005),  who  then 
extended  the  result  to  A  >-  log  m.  □ 


Applying  Lemma  24,  we  have  the  upper  bound  (32  =  0(\/Ao)  under  Pq.  To  lower-bound  (32 
under  P5,  we  simply  use  the  Courant-Fischer  minimax  theorem,  which  implies  that 


82  >  I  :=  min 

ccGM 


lg  +  xlgc)  W(lg  +  xlgc) 

Ills  +  aT^cll2 


where  for  T  C  V  =  {1, . . . ,  N},  1  t  denotes  the  vector  with  l’s  in  coordinates  indexed  by  T,  and 
0’s  elsewhere.  Straightforward  calculations  show  that 

1  W5C  Ws,sc  ■  x  +  A 

-1  = - 1 - mm  -  , 

2  N  —  n  N  —  n  xeM  xz  +  p 
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where 

Ws,s<  =  Y.  wv  ’  A  := 

i£S,j£Sc 

Elementary  calculations  show  that 


W5 


nW gc 


n 


Ws,sc  (N  ~  n)Ws,sc  ’  P  '  N  —  n 


mm 


x  +  A  1 


xeR  x2  +  p  2  A2  +  p  +  A  ’ 


which  is  increasing  in  A.  Noting  that  Ws  ~  Bin(u)2),pi),  while  Wsc  ~  Bin(fV(2),p0)  and  Ws ,  Sc  ^ 
Bin(n(fV  —  n),po),  we  see  that  I  is  stochastically  increasing  with  Ai,  so  that  we  may  assume  that 
Ai/Aq  0.  By  Chebyshev’s  inequality,  under  P 5, 


Ws 


Wsc 

Ws,sc 


n(n  —  l)  , 

- - - pi  +  0(71^)  , 

(N  —  n)(N  —  n  —  1)  ,  _ , 

- - ^ - LPo  +  0{Ny/pd  , 

n(N  -  n)p0  +  O(y/nNp0)  , 


using  the  fact  that  n  =  o(N).  With  this,  we  get 

^  +  0(VpI)  n  ^f^  +  0{N^po) 


A  = 


n(N  —  n)po  +  0(\/nNpo )  N  —  n  n(N  —  n)po  +  0(y/nNpo) 


npi 


2  (N  -  n)p0 


1 


l  +  0(— )  +  0( 


'n^/pi 

,Ai  1 


y/nNp0 


-« U  +  o( 


Ny/PO 


)  +  o( 


VnNp0 


=  ^  +  o(^p)  +  o(:  /—  / 

2Aq  Aq  Aq  nyfp\ 


2 


using  the  fact  that  n  =  o(JV)  and  then  Ai  =  o(Ao). 
1 


1 


- 7  1  - =  UA2  +  p-  A)  =  —  -  1)  +0(1+ 

2yi2T^  +  Al  2/EV  j  2p{\0  >  1 

With  this  and  the  bounds  provided  by  Chebyshev  inequality,  we  find  that 


) 


1 


-I  = 


N  —  n 


Po  +  0{y/po)  +  [np0  +  O(y/np0/N)\ 


N-w-r-P0YL  +  O(np0  +  \j  Npo/n) 
2  Ao 

+  0(nXo/N  +  \J Aq /n)  , 


2p  ^A0  l)+0(l  +  pn%Ao)J 


so  that  ^2  >  Xi+O(n\o/N+  y/\o/  n)  under  P 5.  Comparing  with  the  upper  bound  that  we  obtained 
for  $2  under  Po,  we  conclude  that,  under  the  assumptions  of  Proposition  9,  the  test  that  rejects 
when  >  |Ai  is  asymptotically  powerful.  □ 

We  did  not  explore  in  detail  methods  based  on  the  Laplacian,  or  normalized  Laplacian,  whose 
eigenvalue  perturbation  analysis  is,  for  example,  carried  in  (Chung  and  Radcliffe,  2011). 
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6  Discussion 

6.1  Adapting  to  unknown  p0  and  n 

In  (Arias-Castro  and  Verzelen,  2012),  we  discussed  in  detail  the  case  where  po  is  unknown.  In 
this  situation,  the  total  degree  test  is  not  applicable,  and  we  replaced  it  with  a  test  based  on  the 
difference  between  two  estimates  for  the  degree  variance.  On  the  other  hand,  the  scan  test  (based 
on  (3))  can  be  calibrated  in  various  ways  without  asymptotic  loss  of  power  —  for  example,  by 
plugging  in  the  estimate  po  =  in  place  of  po  •  We  showed  that  a  combination  of  degree  variance 
test  and  the  scan  test  are  optimal  when  po  is  unknown,  so  that  the  degree  variance  test  can  truly 
play  the  role  of  the  total  degree  test  in  this  situation.  We  believe  this  is  the  case  here  also.  In 
addition  to  that,  the  broad  scan  test  (based  on  (6))  can  also  be  calibrated  without  asymptotic  loss 
of  power,  and  the  same  is  true  for  all  the  other  tests  that  we  studied  here,  except  for  the  largest 
connected  component  test  in  the  supercritical  regime. 

We  also  discussed  in  (Arias-Castro  and  Verzelen,  2012)  the  case  where  the  size  of  the  subgraph 
n  is  unknown.  This  only  truly  affects  the  broad  scan  test,  whose  definition  itself  depends  on  n. 
As  we  argued  in  our  previous  paper,  it  suffices  to  apply  the  procedure  to  all  possible  n’s,  meaning, 
consider  the  multiple  test  based  on  a  combination  of  the  statistics 

Wl  n  =  1, . . . ,  N/2 

with  a  Bonferroni  correction.  The  concentration  inequalities  that  we  obtained  for  Wit  can  accommo¬ 
date  an  additional  logarithmic  factor  that  comes  out  of  applying  the  union  to  control  this  statistic 
under  Po,  and  from  this  we  can  immediately  see  that  the  test  is  asymptotically  as  powerful  (up  to 
first  order). 

6.2  Open  problems 

The  cases  where  Ao  — >  0  and  where  liminf  Ao  >  e  are  essentially  resolved.  Indeed,  in  the  first  situa¬ 
tion,  the  largest  connected  component  test  is  asymptotically  optimal  by  Theorem  2  and  Theorem  4 
case  (51),  while  in  the  second  situation  the  broad  scan  test  is  asymptotically  optimal  by  Theorem  1 
and  Theorem  4  cases  (53)  and  (54),  together  with  Theorem  5.  The  case  where  0  <  Ao  <  e  is 
fixed  is  not  completely  resolved.  Since  the  triangle  test  has  non-negligible  power  as  soon  as  Ai  is 
bounded  away  from  0,  consider  r  defined  as  the  largest  real  such  that  no  test  for  G(N,  ^)  versus 
G(N,  y^;n,  — )  is  asymptotically  powerful  when  limsupAi  <  r.  Theorems  2  and  3  provide  some 
upper  bounds  on  r. 

Open  problem  1.  Compute  t  as  a  function  of  Ao  and  k  :=  lirnsup  ^ . 

Although  we  proved  that  the  broad  scan  test  was  asymptotically  optimal  when  liminf  Ao  >  e, 
its  performance  was  described  only  indirectly  in  terms  of  Ai  in  the  case  (13). 

Open  problem  2.  Compute,  as  a  function  of  X\  and  possibly  k  (defined  above),  the  limits  inferior 
and  superior  of 

n  %[Wfc*s] 

sup  - — —  • 

k=n/uN  ^ 

We  also  formulate  an  open  problem  that  connects  directly  with  the  planted  clique  problem.  We 
saw  that  the  broad  scan  test  is  powerful  when  Ai  is  sufficiently  large,  but  we  do  not  know  how  to 
compute  it  in  polynomial  time.  Is  there  a  polynomial-time  test  that  can  come  close  to  that? 

Open  problem  3.  Find  a  polynomial-time  test  that  is  asymptotically  powerful  for  testing  G(N,po) 
versus  G(N,po\n,pi)  when  n2/N  =  0(1),  while  Aq  — >  oo  and  Ai  =  0(1). 
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7  Proofs  of  auxiliary  results 

7.1  Proof  of  Lemma  6 

Fix  e  >  0  and  define  x  :=  2  (1  +  e)  +  y/(l  +  e)2  +  Ai(l  +  e)  .  First,  we  control  the  deviations  of 

W£  s.  Define  qk  =  (Ai  +  x)/(k  —  1)  and  notice  that  qk  >  p\  for  h/un  <  k  <  n.  Since  log(l  +  t)  <t 
for  any  t  >  —1,  we  have 


(i) 


HPI  {qk)  ■■=  Qk  log  —  +  (1  -  qk)  log 


1  ~  Qk 
1  ~Pi 


© 


>  qk  log  —  -  qk  +  pi  ■ 


Applying  an  union  bound  and  Chernoff  inequality  (10),  we  control  the  deviations  of  IF/  s : 


IP’s 


WkS  >  fc(2) 


qk 


< 


n 


where 


.  .  ,eris 

Ak  :=1og(©r)  - 


k 

k-  1 


exp 


-k^Hpi{qk)  <  exp  [kAk\ 


(ftlog(ii) 


k '  2 

Observe  that  x  is  larger  than  2.  As  a  consequence,  we  obtain 


—  }  ~Qk+ Pi 


Ak  =  1  +  log  (  -  )  - 

v  rC  ' 


n 


X\  +  x  (  n(\i  +  x)\  Ai  +  x  X\{k  —  1) 


log 


£  Ai  +  x 

<  1  H - log 

2  2  6 


Ai  +  x 
Ai 


(fc-l)Ai 
Ai 

y 


+ 


k-  1 


n 


-  1  -  log 


2  n 

k-  1 
n 


<  1  - 


ar 


4(Ai  +  .t) 


where  we  used  in  the  last  line  the  inequalities  t  —  logf  —  1  >  0  and  log(l  —t)<—t  —  t2 / 2,  valid  for 
any  t  >  0.  By  definition  of  x,  we  have  x2/(4(Ai  +  x))  =  1  +  e.  In  conclusion,  we  have  proved  that 
for  any  integer  k  between  n/uj\f  and  n 


f5 


wk,s  >  Ai  +  x 
k  ~  2 

ii 


<  exp  [— ke] 


Let  us  now  control  the  lower  deviations  of  using  Lemma  7 


(86) 


Ps 


W, 


k,S 


< 


w, 


k,S 


-  ES 


W*  "i  \  1 /2 
vvk,S 


k1/ 2 


<  T 


For  k  large  enough,  exp  [— ke\  <  1/2,  which  therefore  implies  that 


Es 


k 


<  Fs 


W, 


k,S 


k 


1/2 


{n/uN)1/2 


+ 


Ai  +  x 


since  k  >  h/un .  Taking  the  supremum  over  k  and  letting  n  go  to  infinity,  we  conclude  that 


lim  inf  Eg 

k=n/U]y 


<  lim  inf  —  ^  %  =  lim  inf  ^  +  (1  +  e)  +  \/(l  +  e)2  +  Ai(l  +  e)  . 


Then  letting  e  going  to  zero  allows  us  to  conclude. 
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7.2  Some  combinatorial  results 

We  state  and  prove  some  combinatorial  results. 

Lemma  25  (Extension  of  Cayley’s  identity).  The  number  of  labelled  trees  of  size  £  containing 
a  given  labelled  tree  of  size  k  satisfies 

=  kll~k~x  . 

(g\ 

The  number  T^  ^  of  labelled  trees  of  size  £  containing  a  given  labelled  forest  with  tree  components 
of  size  ki, ...  ,kr  satisfies 


rp(D 

ki,...,kr 


< 


f-k+r-\£  -  k  +  r  -  ly-1 


with  k  =  Yll=  l  ki- 

Proof.  The  proof  relies  on  the  double  counting  argument  of  Pitman  (Aigner  and  Ziegler,  2010). 
Noting  T  the  fixed  tree  of  size  k,  we  count  in  two  ways  the  number  of  labelled  trees  of  size  £  that 
contain  T  and  whose  vertices  outside  T  have  been  ordered.  Straightforwardly,  we  have  T^\i  —  k)\ 
such  trees.  Alternatively,  we  consider  the  following  way  of  building  such  a  labelled  ordered  tree: 

1.  Start  from  T ■ 

2.  Choose  any  vertex  uq  among  the  original  tree  T  and  any  vertex  vq  among  the  (£  —  k)  remaining 
vertices.  Add  an  edge  between  uq  and  vq.  Root  the  given  tree  —  now  of  size  k  +  1  —  at  Do- 
Consider  all  the  £  —  k  —  1  remaining  vertices  as  rooted  trees  of  size  1 . 

3.  Then,  perform  the  iterative  construction  of  Pitman.  At  each  step  i  =  1  ,...,£  —  k  —  1,  add 
an  edge  in  the  following  way:  choose  any  starting  vertex  run  among  the  £  vertices  and  note  pi 
the  root  of  the  tree  containing  m.  Choose  any  ending  vertex  Vi  among  the  {£  —  k  —  i)  roots 
other  than  pt.  This  so-obtained  tree  is  rooted  at  p^. 

4.  Let  denote  the  root  of  the  final  tree. 

All  in  all,  we  have  kmk~]  {£  —  k)\  such  constructions  and  the  sequence  v\, . . .  ,V£-k  obtained  in 
Step  3  provides  an  ordering  for  the  vertices  not  in  T . 

Lemma  26.  For  any  labelled  tree  T  of  size  £  that  contains  T  and  whose  vertices  outside  T  have 
been  ordered,  there  exists  one  and  only  one  construction  of  T  based  on  the  algorithm  above. 

Comparing  the  two  counts  leads  to  the  desired  result. 

Proof  of  Lemma  26.  Let  us  slightly  modify  the  iterative  construction  of  Pitman  by  putting  an 
orientation  on  the  added  edges:  the  first  edge  is  oriented  from  vq  to  uq.  For  any  i  =  1, ...  ,£  —  k  —  1, 
the  edge  between  m  and  Vi  is  oriented  from  m  to  u*.  The  so-obtained  partially  oriented  tree  is 
noted  Tu, v 

Observe  that  except  for  V£_k  which  has  no  parents,  all  other  nodes  Vi  have  one  and  only  one 
parent.  Also,  observe  that  except  for  the  edge  vo  — >  uq,  all  edges  between  nodes  in  the  subtree  T 
and  nodes  in  {ui, . . .  V£-k}  leave  the  subtree  T.  By  a  simple  induction,  this  leads  us  to  the  following 
claim: 
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Claim  1:  All  partially  oriented  tree  Tu,v  based  on  Pitman  construction  with  sequences  (u,  v)  = 
(h0,  ho,  ui,  ■  ■  ■ ,  U£-k- i,vi,  ■  ■  • ,  V£-k-i,ve-k)  satisfy  the  following  property 


Any  edge  in  T  is  undirected, 

Any  edges  on  the  unique  path  between  V£_k  and  T  is  oriented  towards  T, 
Any  other  edge  (not  in  T)  is  oriented  in  the  opposite  direction  to  T. 


(P) 


In  fact,  this  property  characterizes  the  oriented  partially  trees  Tu,v 

Claim  2:  Conversely,  for  any  sequence  v  =  (v\, . . . ,  vg_k)  and  any  partially  oriented  tree  7^  of  size 
£  satisfying  (P),  there  exists  a  unique  sequence,  (ho,  ho,  u\, . . . ,  ui-k-i)  such  that  7^ u,v  =  T ■ 

Proof  of  Claim  2.  The  uniqueness  is  straightforward.  Given  7^,  define  ho  as  the  unique  child 
in  T  and  define  ho  the  parent  of  ho-  For  any  i  =  1,  ...,£  —  k  —  1,  denote  u\  the  parent  of  u* .  These 
sequences  are  lawful  for  the  Pitman  construction.  Indeed,  at  step  i,  Vi  is  not  in  the  same  connected 
component  as  Ui  and  is  still  a  root  of  a  connected  component. 

Then,  the  lemma  proceeds  from  the  fact  that  for  any  tree  T  of  size  £  and  any  sequence  v  = 
(vi, . . .  ,ve-k),  there  exists  one  and  only  partially  orientation  of  T  satisfying  (P).  □ 

We  now  prove  the  second  part  of  Lemma  25,  relying  on  the  same  double  counting  argument. 
Write  k  =  ki  +  . . .  +  kr.  Noting  P  the  fixed  forest  with  (labelled)  connected  components  71, . . . ,  Tr 
of  respective  sizes  k\ , . . . ,  kr ,  we  count  in  two  ways  the  number  of  labelled  trees  of  size  t  that  contain 
T  and  whose  vertices  outside  T  have  been  ordered.  Straightforwardly,  we  have  T kr(£  —  k)\  such 
trees.  Alternatively,  we  consider  the  following  Pitman  construction: 

1.  Start  from  T . 

2.  For  any  j  =  1 , ,r,  choose  any  vertex  Wj  6  Tj.  Root  Tj  at  Wj.  Consider  all  the  i  —  k 
remaining  vertices  as  rooted  trees  of  size  1. 

3.  Then,  perform  the  iterative  construction  of  Pitman:  at  each  step  i  =  1  —  k  +  r  —  1, 

add  an  edge  in  the  following  way:  choose  any  starting  vertex  Ui  among  the  t  vertices  and 
note  pi  the  root  of  the  tree  containing  m.  Choose  any  ending  vertex  Vi  among  the  remaining 
{£  —  k  +  r  —  i)  roots  other  than  p,;.  The  resulting  tree  is  rooted  at  p*. 

4.  Let  vi-k+r  denote  the  root  of  the  final  tree. 

All  in  all,  we  have  {I  —  k  +  r  —  1)!  such  constructions.  And  the  sequence 

vi, . . .  ,V£_k+r  obtained  in  Step  3  provides  an  ordering  of  the  vertices  outside  T  if  we  ignore  the 
Wj’ s  in  that  sequence. 

Lemma  27.  Any  tree  that  contains  T  and  whose  vertices  outside  T  are  ordered  by  the  sequence 
(ti, . . . ,  t£-k)  can  be  constructed  in  this  way. 

Consequently  we  have 


r 


Tlf. k)<  <  (llki)  e 


i£—k+r— 1 


(£  —  k  +  r  —  1)! 


2—1 


from  which  we  derive  the  (crude)  bound 


□ 
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Proof  of  Lemma  21.  Consider  a  tree  T *  that  contains  T  and  whose  vertices  outside  T  are  ordered 
in  the  following  sequence  (ti, . . .  ,t(-k)- 

Claim.  There  exists  a  (non-necessarily  unique)  orientation  of  the  edges  outside  J~  such  that  any 
node  in  ti, . . . ,  tg-k-i  has  exactly  one  parent,  fy-k  has  no  parent,  and  any  tree  7}  in  T  has  exactly 
one  parent. 

Proof  of  Claim  1:  Collapse  each  of  the  trees  71  into  a  single  node,  to  obtain  the  tree  ‘P(--k+r 
with  i  —  k  +  r  nodes.  Then,  we  prove  the  result  for  p^k+r  by  a  simple  induction  on  the  number 
of  nodes. 

For  any  i  E  {l,...,r},  define  Wj  as  the  unique  node  in  7).  We  define  the  sequence  v  := 

(cui, . . . ,  ujr,ti, . . .  tg _ k) .  Finally,  we  define  Ui  as  the  unique  parent  of  V{  for  any  i<£  —  k  +  r—  1.  It 

is  straightforward  to  check  that  these  sequences  oj,  v  and  u  are  lawful  for  the  Pitman  construction 
and  allow  to  build  P .  □ 

7.3  Proof  of  Lemma  14 

By  definition, 

Var0[lVfcree]  =  (Po[f5ci  and  Gc2  are  trees]  -  Po[<5ci  is  a  tree]Po [Gc2  is  a  tree])  , 

Ci,C2 

where  the  sum  ranges  over  subsets  C\ ,  C-2  of  size  k. 

In  the  sequel,  we  let  q  =  |Ci  C  C2I  and  let  r  denote  the  number  of  connected  components  of 
C\  n  C2 .  Note  that,  when  <7  =  0,  the  corresponding  terms  in  the  sum  above  are  zero.  When  q  >  1, 
we  define 


Br  q  =  Pq  \Gc\  ■.  Gc2  are  trees  and  GcinC2  has  r  connected  components]  , 


P0[Sci  and  Gc2  are  trees]  =  ^  Br,q  . 

r= 1 

Note  that  GcinC2  is  a  forest  when  Gc2  and  Gc2  are  trees. 

We  derive  Bi>q  first.  Under  the  event  {Gci  ■  Gc2  and  Gc\ rC2  are  trees},  there  are  exactly  2k—l—q 
edges  in  Gc\  U  Gc2  among  the  potential  2 k^  —  gC)  edges.  Let  us  count  the  number  of  configurations 
compatible  with  this  event.  By  Cayley’s  identity,  there  are  qq~2  configurations  for  the  tree  GcinC2- 
The  tree  GcxnC2  being  fixed,  we  apply  Lemma  25  to  derive  that  there  are  qkk~q~1  configurations 
for  Gci  and  qkk~q~l  configurations  for  Gc2-  All  in  all,  we  get 


Bl,q 


qq~2[qkk~g-1]2pQk~1~q (1  —  p0)2fc(2)-l?(2)-2fe+i+9  . 


Then,  we  upper  bound  BVtq  for  r  >  2.  Under  the  event  defined  in  Br^q,  there  are  2k  —  2  —  q  +  r 
edges  in  Gci  U  Gc2  among  the  potential  2 k^  —  q^  edges.  By  Lemma  19  ,  there  are  less  than  qq~2 
conhgurations  for  the  forest  GcxnC2  with  r  connected  components.  GcxnC2  being  fixed,  Lemma  25 
tells  us  that  there  are  less  than  (2)r  kk~q+r~1  (k  —  q  +  r  —  l)r_1  possible  conhgurations  to  complete 
Gc-i  and  (independently)  for  Gc2  ■  It  then  follows  that 


Br,q  <  qq~ 2 


kk-q+r~\k  -  q  +  r  -  I)""1]  2  Plk~q+r~2{l  -  pofkP)-qP)-2k+q-r+2 


PO 


1  -Po 


r— 1 


r6r— 4 
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using  the  fact  that  q  <  k.  Summing  over  r  leads  to 
9  9 

^2  Br,q  <  Bkq  k 2  ^2 


Pok 6  T  1  /  p  Pok8  _  ,  x 


r=2  t=2<-1~P° 

since  pok8  =  o(l).  Thus,  when  \C\  fl  C2I  =  q  >  1,  we  obtain 

P0[£ci  and  0C2  are  trees]  =  Sii9  +  o(^ii9) 

X  qqk2k~2q~2pQk~l  q  . 

We  can  now  bound  the  variance.  The  number  of  subsets  (C±,  C2)  of  size  k  such  that  C\  (IC2  =  q 
equals  {2)  {t)  Clin) ■  Thus,  we  derive 


VaroK 


treen 


k 

E 

9=1 


N  -k 

k  —  q 


qQk2k-2q-2p2k-q-l 


k  nqu2k-2q-2 
V-  x  2fc-2g-l 

^  q\(k-q)l2  0 

(X0e)k  .  .  _  (ky/%e\ 

q=  1  x 


by  Stirling’s  lower  bound.  By  convention,  Aq  =  1.  The  function  £  — >  is  easily  seen  to  be 

increasing  over  (0,  ky/Xo/e)  and  decreasing  over  (k^Wz  ,00).  Thus,  when  Ao  <  e,  we  have 
Ak-q  <  and  when  Ao  >  e,  we  have  Ak-q  <  Hp  this  is  for  all  <7  =  1, . . . ,  k.  Then  summing 

over  q,  we  obtain  the  stated  bounds  in  each  case. 


7.4  Proof  of  Lemma  15 

First,  we  deal  with  the  expectation. 

%k:s„]  =  e  E  Ps[£ci  and  ^CiuC2  are  trees]  . 

CiCS,|Ci|=q  C2CSc,\C2\=k-q 

When  Qc]  and  Qc\  uC2  are  both  trees,  there  are  q  —  1  edges  in  Qcx  and  k  —  q  additional  edges  in 
Qc\ jC2 ■  The  number  of  configurations  for  Gc\  is  qq~2  (Cayley’s  Identity).  By  Lemma  25,  when 
Gci  is  fixed,  there  remains  qkk~q~3  possible  configurations  for  GcpjC2-  As  for  the  previous  variance 
computation,  we  apply  to  control  this  probability.  Hence,  we  get 

P5 [Gci  and  QciUC2  are  trees]  =  qq^2qkk~q^1p\~1pQ~q(l  -  pi)q(2) ~q+1  (1  -  p0)ki2)~qi2)~k+q 

y  qq-1kk-q-1pq-1pk0-q  , 

since  ( q ^  —  q  +  l)p\  <  k2pi  x  k2/n  =  o(l)  and  ( k ^  —  q ^  —  k  +  q)po  <  k2po  x  k2/N  =  o(l). 
Hence,  using  the  fact  that  nk  =  o(N )  and  the  usual  bound  ml  <  \/m(m/e)m,  and  we  derive 

qq-lkk-q-lXq-lXk-q 

^  "  q'-(k-q)\ 

(eAx)fc  f  Ap fc  A 

U  Xik3  \X i(k-q)J 


k—q 
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This  quantity  is  maximized  with  respect  to  q  when  ( k  —  q)/k  =  Ao/(A4e),  and  taking  q  :=  k— 
leads  to 

Esl"w.Jv"17p-exp(^*:)  ' 

Let  us  turn  to  the  variance.  Again,  we  decompose  it  as  a  sum  over  (Ci,  C2)  C  S 2  and  (C3,  64)  C 
(5C)2  depending  on  the  sizes  s  =  \C\  n  C2\  and  r  =  1 63  n  C  4 1 .  By  independence  of  the  edges,  only 
the  subsets  such  (r,  s)  7^  (0,  0)  play  a  role  in  the  variance.  We  have 

q  k-q  _ 

VarS[JVfs,,]  <  EE  E  E  IPs[£ci,  Sc2,  ^CiUC3)  Gc2uc4  are  trees] 

S=1  r=0  |CinC2|=s  |C3nC4|=r 
k—q 

+E  E  E  IPs [f?Ci ,  <?c2,  ^CiuC3,  £c2uc4  are  trees] 

r= 2  |CinC2|=0  |C3nC4|=r 
=  -Bi  +  B2  . 


First  we  consider  the  sum  Bi  where  r  is  positive.  Therefore,  fix  Ci ,  C2  C  S  and  C  Sc  with 

I  Cl  I  =  |U2|  =  q,  IC3I  =  IC4I  =  k  —  q,  |  Ci  n  C2\  =  s  >  1  and  IC3  PI  C4I  =  r  >  1,  and  for  1  <  t\  <  s 
and  1  <  t2  <  r  +  s,  define 


Ad, *2)  :=  {GCl,  Gc2,  Gc4uC3,  Gc2uC4  are  trees} 

fl  {Gc4nC2  has  t\  connected  components} 
n  { G{c4 nC2 )u(C3nC4)  has  t2  connected  components}  . 

(The  dependency  of  Aitl,t2)  011  Ci,  C2,  C3,  C4  is  left  implicit.) 

We  first  control  Ps[.A(i,i)]-  Under  the  event  A(i .4),  the  graph  Qqx  U  Gc2  contains  2q  —  1  —  s 
edges  and  the  graph  GciuC3  U  f?c2uC4  contains  2 (k  —  q)  —  r  additional  edges.  Indeed,  the  number 
of  edges  in  the  last  graph  is  equal  to 


(|C1UC3|-l)  +  (|C2UC4|-l)-(|(C1nC2)U(C3nG4)|-l)  =  k-l  +  k-l-(r+s-l)  =  2k-r-s-l 


Applying  Lemma  25,  there  are  ss~2  possible  configurations  for  GcirC2  and  then  [sg9-5-1]2  possible 
configurations  to  complete  Gc\  U  Gc2 ■  The  graph  Qcx  U  Gc2  been  fixed,  there  are  s(s  +  r)r^1 
configurations  for  ^(CinC2)u(C3nC4)i  since  this  is  a  tree  with  s  +  r  nodes  containing  the  given  tree 
Gc\ rC2  with  s  nodes.  By  the  same  token,  Gc4uC3  is  a  tree  with  k  nodes  that  includes  the  given  tree 
^OiU(C3nC4)  with  q  +  r  nodes,  and  similarly  for  Gc2uC4,  there  at  most  [(q  +  r)kk^q~r~1]2  possible 
configurations  to  complete  Gc4\jC3  U  Gc2uC4-  Thus,  we  obtain 


PstAi,!)]  <  ss-2[sqq-s-l}2s{s  +  ry-l[(q  +  r)kk-q-r 


2q-l-s  2(k-q)~r 

\  Pi  P 0 


Ai,i  . 


(87) 


Let  us  now  control  the  probability  of  A(t  ljt2)  for  t\  or  t2  strictly  larger  than  one.  First,  observe 
that  whenever  t2  <  t\.  Atltt2  is  empty.  Indeed,  if  t2  <  t%,  there  is  a  path  in  C3  n  C4  between  two 
connected  components  of  Gc\  rC2  •  However,  these  two  connected  components  are  also  related  by  a 
different  (since  C\  n  C3  =  0)  path  in  C\  (since  Gci  is  a  tree),  and  that  contradicts  the  fact  that 
Gc4uC3  is  a  tree.  Hence,  we  may  assume  that  t2  >  t\ .  By  Lemmas  25  and  19,  there  are  at  most 
ss~2  possible  configurations  for  the  forest  Gc4rC2,  and  when  this  is  fixed,  there  are  at  most 

[(s/ti)Vs+tl_1(g  ~s  +  t  1  -  l)*1'1]2  <  [sqi-SqW'-V]2 
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possible  configurations  to  complete  Qcx  U  Qc2  ■  With  Qa  U  Qc2  being  fixed,  the  number  of  possible 
configurations  for  £?(CinC2)u(C3nC4)  is  at  most  the  number  of  trees  that  contain  GcinC2  —  which  is 
at  most 

(s/t if1  (s  +  ry+r-s+tl-\s  +  r-s  +  h-  l)tl_1  <  s*1  (s  +  r)r+2tl~2 

by  Lemma  25  —  times  kt2~l,  which  bounds  the  number  of  ways  of  erasing  ^2  —  1  edges  in  this 
tree  to  obtain  a  forest  with  t2  components.  The  graph  <5ciU(C3nc4)  contains  t2  —  t\  +  \  connected 
components.  By  Lemma  25,  there  are  no  more  than 

2 

<  [(q  +  r)kk~r~Qk3('t2~tl')}2 

possible  configurations  to  complete  GciuC3  U  <5c2uC4-  The  number  of  edges  in  Gch  U  Qc2  is  2 (q  — 
1)  -  (s  -  t\),  while  the  number  of  edges  in  ^CiuC3  U  Gc2uC4  is 

2 (k  -  q)  -  (r  -  (t2  -  h))  =  2 (k  -  q)  -  r  +  t2  -  h  . 


q  +  r 


t2-t  i  +  l 


t2—ti+i 


kk-q-r+t2-tl  (k_q_r  +  t2_ 


All  together,  and  with  some  elementary  simplifications,  we  arrive  at  the  following  bound 

Since  k0^\ po  +  pi)  =  o(l),  it  follows  that 

s  r+s 


yi  ps(Ai,t2)  -< 

tl=l*2=l 


1- 


Using  the  definition  of  A\  \  in  (87)  and  the  definition  of  q,  we  bound  B\ 

q  k-q 


Bi  < 

S=1 

-< 

n'z 

& 

-< 

n  ^ 

r 

-< 

n 

X\ 

-< 

n 

Ai 

n\  ( N  —  n\  ( q\  (  k  —  q\  (  n  —  q\  ( N  —  n  —  k  +  q 


qj  \  k  —  q  J  \s 


q-s 


Ai 


ss~^^  q2^  s  L  (g  -g  By  ^  k2^k  ^  U  2  (q  t*) 


r\s\(q  —  s)\2(k  —  q  —  r)\2 

n2k-r-s  (  S  +  r 


k  —  q  —  r 

*  ^2q—s—l  yi(k—q)—r 


.  2 (q—s)  /  1  \  2 (k—q—r) 

Q  \  (  k  \  ’  y2q-s-ly2{k-q)-r+l 

k—q—r )  1  0 


q-s 


E 

r,s 

E 


=4fc— 2q— 3r—  s  /  S  +  T 


q-s 


2(q-s) 


04k—2q—3r  I  _ 9_ 


q-s 


2  (q-s) 


k  —  q 
k  —  q  —  r 


k  —  q 
k  —  q  —  r 

2  (k—q—r) 


2  (k—q—r) 


\2k—2r—s—l  \r 
A1  A0 


\2k—2r—s—l  \r 
A1  A0  * 


We  have  applied  Stirling’s  lower  bound  in  the  fourth  line;  we  haved  used  the  definition  of  q  to 
control  k/(k  —  q)  in  the  fifth  line 


k  \  2 (k-q-r) 


k  —  q 


k 


2  (k—q—r) 


I  AoL  I 

L  Aie-I 


Xie\2(k-q-r)  (_  AieN  ~2k 


~  '  A0  ) 


1  - 


k\o 


2  (k—q—r) 
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and  we  have  upper-bounded  (1  +  s/r)r  by  es  in  the  last  line.  Note  that 


„-3  r 


k  —  q 
k  —  q  —  r 


2  (k—q—r) 


\ —2 r \r  _ 

A1  Ao  — 


(t-^3/2^2|t^’Y3,2  a, 

k—q—r  )  V 


is  decreasing  with  respect  to  r  since  A}e  >  Aq.  As  a  consequence,  we  have 


nk 


t=  i 


The  function  l  — >  Dp  is  easily  seen  to  be  maximized  at  (  =  q\fX[/e.  This  allows  us  to  conclude 
that 

_g  (Ak-2q+2q^/X[/e\2k-q 

Ai 

Finally,  we  bound  B2  following  a  similar  strategy.  First,  we  observe  that  the  probability  of  the 
event  B  :=  {Qc\i  Gc2i  Gc4uC3->  Gc2uC4  are  trees}  is  equivalent  to  the  probability  of  the  event  B\  := 
Bn{Qc3nC4  is  a  tree}.  This  follows  from  the  fact  that  the  event  Br  :=  Bf]{Qc3nC4  contains  r  trees} 
involves  r  —  1  more  edges  than  B\  while  the  number  of  possible  configurations  in  Br  is  does  not 
increase  more  than  by  a  factor  k°G)r  compared  to  B\. 


k—q 

b2  =  EE  E  IP’s  \QC\ ,  Gc2,  GciuC3,  Gc2uC4  are  trees] 
r  2  |CinC2|=0  \c3nc4\=r 

k—q 

-EE  E  Ps[Sci,  Gc2,  Gc4uC3,  Gc2uC4  ,  and  Qc3r\C4  are  trees] 

r=2  |CinC2|=0  \C3nC4\=r 


^Af-2A0e4fc-2"  . 

In  the  third  line,  we  bound  the  probability  by  counting  the  number  of  edges  involved  in  the  event 
and  the  number  of  possible  configurations,  as  we  did  before.  In  the  fourth  line,  we  use  the  bound  of 
k/(k  —  q )  to  obtain  a  ratio  of  the  form  .  In  the  last  line,  we  observe  that  the  sum  is  decreasing 

with  respect  to  r  and  is  maximized  at  r  =  0. 


Acknowledgements 

We  would  like  to  thank  Jacques  Verstraete  and  Raphael  Yuster  for  helpful  discussions  and  refer¬ 
ences  on  counting  fc-cycles.  The  research  of  N.  Verzelen  is  partly  supported  by  the  french  Agence 


56 


Nationale  de  la  Recherche  (ANR  2011  BS01  010  01  projet  Calibration).  The  research  of  E.  Arias- 
Castro  is  partially  supported  by  a  grant  from  the  Office  of  Naval  Research  (N00014-13-1-0257). 


References 

Aigner,  M.  and  G.  M.  Ziegler  (2010).  Proofs  from  The  Book  (Fourth  ed.).  Berlin:  Springer- Verlag. 

Alon,  N.  (1998).  Spectral  techniques  in  graph  algorithms  (invited  paper).  In  LATIN’98:  theoretical 
informatics  (Campinas,  1998),  Volume  1380  of  Lecture  Notes  in  Comput.  Sci.,  pp.  206-215. 
Berlin:  Springer. 

Alon,  N.  and  S.  Gutner  (2010).  Balanced  families  of  perfect  hash  functions  and  their  applications. 
ACM  Trans.  Algorithms  6(3),  Art.  54,  12. 

Alon,  N.,  M.  Krivelevich,  and  B.  Sudakov  (1998).  Finding  a  large  hidden  clique  in  a  random  graph. 
In  Proceedings  of  the  Eighth  International  Conference  “Random  Structures  and  Algorithms’' 
(Poznan,  1997),  Volume  13,  pp.  457-466. 

Alon,  N.,  R.  Yuster,  and  U.  Zwick  (1997).  Finding  and  counting  given  length  cycles.  Algorith- 
mica  1 7(3),  209-223. 

Arias-Castro,  E.  and  N.  Verzelen  (2012).  Community  detection  in  dense  random  networks.  Available 
online  at  http://arxiv.org/abs/1302.7099. 

Arsic,  B.,  D.  Cvetkovic,  S.  K.  Sirnic,  and  M.  Skaric  (2012).  Graph  spectral  techniques  in  computer 
sciences.  Appl.  Anal.  Discrete  Math.  6(1),  1-30. 

Berthet,  Q.  and  P.  Rigollet  (2012).  Optimal  detection  of  sparse  principal  components  in  high 
dimension.  Available  online  at  http://arXiv.org/abs/1202.5070. 

Bickel,  P.  J.  and  A.  Chen  (2009).  A  nonparametric  view  of  network  models  and  newmanaigirvan 
and  other  modularities.  Proceedings  of  the  National  Academy  of  Sciences  106(50),  21068-21073. 

Bollobas,  B.  (2001).  Random  graphs  (Second  ed.),  Volume  73  of  Cambridge  Studies  in  Advanced 
Mathematics.  Cambridge:  Cambridge  University  Press. 

Boucheron,  S.,  O.  Bousquet,  G.  Lugosi,  and  P.  Massart  (2005).  Moment  inequalities  for  functions 
of  independent  random  variables.  Ann.  Probab.  33(2),  514-560. 

Butucea,  C.  and  Y.  I.  Ingster  (2011).  Detection  of  a  sparse  submatrix  of  a  high-dimensional  noisy 
matrix.  Available  from  http://arxiv.org/abs/1109.0898. 

Chung,  F.  and  M.  Radcliffe  (2011).  On  the  spectra  of  general  random  graphs.  Electron.  J.  Corn- 
bin.  18(1),  Paper  215,  14. 

Chung,  F.  R.  K.  (1997).  Spectral  graph  theory,  Volume  92  of  CBMS  Regional  Conference  Series  in 
Mathematics.  Published  for  the  Conference  Board  of  the  Mathematical  Sciences,  Washington, 
DC. 

d’Aspremont,  A.,  L.  El  Ghaoui,  M.  I.  Jordan,  and  G.  R.  G.  Lanckriet  (2007).  A  direct  formulation 
for  sparse  pea  using  semidefinite  programming.  SIAM  Review  49(3),  434-448. 

Dekel,  Y.,  O.  Gurel-Gurevich,  and  Y.  Peres  (2011).  Finding  hidden  cliques  in  linear  time  with  high 
probability.  In  Proceedings  of  the  Eighth  Workshop  on  Analytic  Algorithmics  and  Combinatorics 
(ANALCO),  pp.  67-75.  Society  for  Industrial  and  Applied  Mathematics  (SIAM). 

Feige,  U.  and  E.  Ofek  (2005).  Spectral  techniques  applied  to  sparse  random  graphs.  Random 
Structures  Algorithms  27(2),  251-275. 

Feige,  U.  and  D.  Ron  (2010).  Finding  hidden  cliques  in  linear  time.  In  21st  International  Meeting  on 
Probabilistic,  Combinatorial,  and  Asymptotic  Methods  in  the  Analysis  of  Algorithms  (AofA’10), 
Discrete  Math.  Theor.  Comput.  Sci.  Proc.,  AM,  pp.  189-203.  Assoc.  Discrete  Math.  Theor. 
Comput.  Sci.,  Nancy. 

Fortunato,  S.  (2010).  Community  detection  in  graphs.  Physics  Reports  486(3-5),  75  -  174. 


57 


Fiiredi,  Z.  and  J.  Komlos  (1981).  The  eigenvalues  of  random  symmetric  matrices.  Combinator¬ 
ial  1(3),  233-241. 

Girvan,  M.  and  M.  E.  J.  Newman  (2002).  Community  structure  in  social  and  biological  networks. 
Proceedings  of  the  National  Academy  of  Sciences  99(12),  7821-7826. 

Heard,  N.  A.,  D.  J.  Weston,  K.  Platanioti,  and  D.  J.  Hand  (2010).  Bayesian  anomaly  detection 
methods  for  social  networks.  Ann.  Appl.  Stat.  4( 2),  645-662. 

Hoeffding,  W.  (1956).  On  the  distribution  of  the  number  of  successes  in  independent  trials.  Ann. 
Math.  Statist.  27,  713-721. 

Janson,  S.  (2005).  The  first  eigenvalue  of  random  graphs.  Combin.  Probab.  Comput.  lf( 5-6), 
815-828. 

Krivelevich,  M.  and  B.  Sudakov  (2003).  The  largest  eigenvalue  of  sparse  random  graphs.  Combin. 
Probab.  Comput.  12(1),  61-72. 

Lancichinetti,  A.  and  S.  Fortunato  (2009,  Nov).  Community  detection  algorithms:  A  comparative 
analysis.  Phys.  Rev.  E  80,  056117. 

Lehmann,  E.  L.  and  J.  P.  Romano  (2005).  Testing  statistical  hypotheses  (Third  ed.).  Springer 
Texts  in  Statistics.  New  York:  Springer. 

Maslov,  S.,  K.  Sneppen,  and  A.  Zaliznyak  (2004).  Detection  of  topological  patterns  in  complex 
networks:  correlation  profile  of  the  internet.  Physica  A:  Statistical  Mechanics  and  its  Applica¬ 
tions  333 ( 0),  529  -  540. 

Mongiovi,  M.,  P.  Bogdanov,  R.  Ranca,  E.  E.  Papalexakis,  C.  Faloutsos,  and  A.  K.  Singh  (2013). 
Netspot:  Spotting  significant  anomalous  regions  on  dynamic  networks.  In  SIAM  International 
Conference  on  Data  Mining. 

Mossel,  E.,  J.  Neeman,  and  A.  Sly  (2012).  Stochastic  block  models  and  reconstruction.  Available 
from  http :  //http :  //arxiv .  org/ abs/ 1202 . 1499. 

Newman,  M.  E.  J.  (2006).  Modularity  and  community  structure  in  networks.  Proceedings  of  the 
National  Academy  of  Sciences  103(23),  8577-8582. 

Newman,  M.  E.  J.  and  M.  Girvan  (2004,  Feb).  Finding  and  evaluating  community  structure  in 
networks.  Phys.  Rev.  E  69,  026113. 

Park,  Y.,  C.  Priebe,  and  A.  Youssef  (2013).  Anomaly  detection  in  time  series  of  graphs  using  fusion 
of  graph  invariants.  Selected  Topics  in  Signal  Processing,  IEEE  Journal  of  7(1),  67-75. 

Pittel,  B.  and  N.  C.  Wormald  (2005).  Counting  connected  graphs  inside-out.  Journal  of  Combina¬ 
torial  Theory,  Series  B  93(2),  127  -  172. 

Pothen,  A.,  H.  D.  Simon,  and  K.-P.  Liou  (1990).  Partitioning  sparse  matrices  with  eigenvectors 
of  graphs.  SIAM  J.  Matrix  Anal.  Appl.  11(3),  430-452.  Sparse  matrices  (Gleneden  Beach,  OR, 
1989). 

Reichardt,  J.  and  S.  Bornholdt  (2006,  Jul).  Statistical  mechanics  of  community  detection.  Phys. 
Rev.  E  74,  016110. 

Robinson,  R.  W.  and  N.  C.  Wormald  (1992).  Almost  all  cubic  graphs  are  Hamiltonian.  Random 
Structures  Algorithms  3(2),  117-125. 

Robinson,  R.  W.  and  N.  C.  Wormald  (1994).  Almost  all  regular  graphs  are  Hamiltonian.  Random 
Structures  Algorithms  5(2),  363-374. 

Rukhin,  A.  and  C.  E.  Priebe  (2012).  On  the  limiting  distribution  of  a  graph  scan  statistic.  Com¬ 
munications  in  Statistics- Theory  and  Methods  ^7(7),  1151-1170. 

Sun,  X.  and  A.  B.  Nobel  (2008).  On  the  size  and  recovery  of  submatrices  of  ones  in  a  random 
binary  matrix.  J.  Mach.  Learn.  Res.  9,  2431-2453. 

Takacs,  L.  (1988).  On  the  limit  distribution  of  the  number  of  cycles  in  a  random  graph.  J.  Appl. 
Probab.  (Special  Vol.  25A),  359-376.  A  celebration  of  applied  probability. 

Van  der  Hofstad,  R.  (2012).  Random  graphs  and  complex  networks.  Available  online  at  http: 


58 


//www . win . tue . nl/~rhof  stad/NotesRGCN . pdf . 

Wang,  B.,  J.  M.  Phillips,  R.  Schreiber,  D.  M.  Wilkinson,  N.  Mishra,  and  R.  Tarjan  (2008).  Spatial 
scan  statistics  for  graph  clustering.  In  SIAM  International  Conference  on  Data  Mining. 
Wormald,  N.  C.  (1999).  Models  of  random  regular  graphs.  In  Surveys  in  combinatorics,  1999 
(Canterbury),  Volume  267  of  London  Math.  Soc.  Lecture  Note  Ser .,  pp.  239-298.  Cambridge: 
Cambridge  Univ.  Press. 


